from:"GWicke \(Code Review\)"

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Config: Temporarily block ceb.wikipedia.org

2017-07-06 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/363645 )

Change subject: Config: Temporarily block ceb.wikipedia.org
..


Config: Temporarily block ceb.wikipedia.org

Recently ceb wiki protected all of tehir templates which made
change-prop accumulate an enormous backlog for that wiki.

Sadly, the if-modified-since optimisation didn't work as
it clashed with the no-content-change optimisation (already fixed
for the future for this case).

To quickly reduce the backlog let's just ignore the ceb.wiki for some
time, it's almost entirely bot generated, also these reparses are no-op.

Change-Id: I31dde948f4c49251e3d594f8d2ea29778c5cd236
---
M scap/templates/config.yaml.j2
1 file changed, 6 insertions(+), 0 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2
index 7ffa612..d5b32f4 100644
--- a/scap/templates/config.yaml.j2
+++ b/scap/templates/config.yaml.j2
@@ -335,9 +335,15 @@
 x-restbase-mode: '{{message.tags[1]}}'
   query:
 redirect: false
+match_not:
+  meta:
+domain: ceb.wikipedia.org
   - match:
   meta:
 schema_uri: 'continue/1'
+match_not:
+  meta:
+domain: ceb.wikipedia.org
 exec:
   method: post
   uri: 
'/sys/links/transcludes/{message.original_event.page_title}'

-- 
To view, visit https://gerrit.wikimedia.org/r/363645
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I31dde948f4c49251e3d594f8d2ea29778c5cd236
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/change-propagation/deploy
Gerrit-Branch: master
Gerrit-Owner: Ppchelko 
Gerrit-Reviewer: GWicke 
Gerrit-Reviewer: Mobrovac 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Set up grafana dashboard monitoring for services

2017-06-30 Thread GWicke (Code Review)

GWicke has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/362567 )

Change subject: Set up grafana dashboard monitoring for services
..

Set up grafana dashboard monitoring for services

Based on
https://github.com/wikimedia/puppet/commit/401973ab7f79fd4567749fe074ccce1d47446581.

Bug: T162765
Change-Id: I5738b379be523341ed5f3aedca1b237ba4ca63cf
---
A modules/icinga/manifests/monitor/services.pp
M modules/role/manifests/icinga.pp
2 files changed, 21 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/67/362567/1

diff --git a/modules/icinga/manifests/monitor/services.pp 
b/modules/icinga/manifests/monitor/services.pp
new file mode 100644
index 000..af36181
--- /dev/null
+++ b/modules/icinga/manifests/monitor/services.pp
@@ -0,0 +1,20 @@
+# == Class: icinga::monitor::performance
+#
+# Monitor Performance
+class icinga::monitor::services {
+monitoring::grafana_alert { 'db/restbase':
+contact_group   => 'team-services',
+}
+
+monitoring::grafana_alert { 'db/api-summary':
+contact_group   => 'team-services',
+}
+
+monitoring::grafana_alert { 'db/services-alerts':
+contact_group   => 'team-services',
+}
+
+monitoring::grafana_alert { 'db/eventbus':
+contact_group   => 'team-services',
+}
+} 
diff --git a/modules/role/manifests/icinga.pp b/modules/role/manifests/icinga.pp
index ead3a61..c87a027 100644
--- a/modules/role/manifests/icinga.pp
+++ b/modules/role/manifests/icinga.pp
@@ -22,6 +22,7 @@
 include icinga::monitor::elasticsearch
 include icinga::monitor::wdqs
 include icinga::monitor::performance
+include icinga::monitor::services
 include icinga::monitor::reading_web
 include icinga::event_handlers::raid
 

-- 
To view, visit https://gerrit.wikimedia.org/r/362567
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5738b379be523341ed5f3aedca1b237ba4ca63cf
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Restart pdfrender service once per day

2017-06-19 Thread GWicke (Code Review)

GWicke has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/359967 )

Change subject: Restart pdfrender service once per day
..

Restart pdfrender service once per day

Temporary work-around for hang issues. A more permanent fix will be
implemented once a decision on the longer term browser-based render
service has been made.

Bug: T159922
Change-Id: I2b456cfc69a56ccc978706b4149e7457ba77ff22
---
M modules/pdfrender/manifests/init.pp
1 file changed, 7 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/67/359967/1

diff --git a/modules/pdfrender/manifests/init.pp 
b/modules/pdfrender/manifests/init.pp
index 32cae42..7b21383 100644
--- a/modules/pdfrender/manifests/init.pp
+++ b/modules/pdfrender/manifests/init.pp
@@ -147,4 +147,11 @@
 systemd=> true,
 service_params => $params,
 }
+
+# Ugly work-around: Restart pdfrender service once per day. See T159922.
+cron { 'periodic_pdfrender_restart':
+command => '/usr/sbin/service pdfrender restart >/dev/null 
2>/dev/null',
+hour=> 5,
+minute  => fqdn_rand(60, 'periodic_pdfrender_restart'),
+}
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/359967
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2b456cfc69a56ccc978706b4149e7457ba77ff22
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Limit Electron memory usage to 2G

2017-06-13 Thread GWicke (Code Review)

GWicke has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/35 )

Change subject: Limit Electron memory usage to 2G
..

Limit Electron memory usage to 2G

In normal operation, Electron uses significantly less memory, but there
have been instances where it started to use up to 15G. Since memory on
SCB nodes is currently very tight (ORES using most of it), this impacted
other services. It is better to avoid such impacts. Ideally, the service
would exit & restart.

Bug: T167834
Change-Id: If199a0271b7d36e26a3dd90e9c130fd71eb78945
---
M modules/pdfrender/templates/initscripts/pdfrender.systemd.erb
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/88/35/1

diff --git a/modules/pdfrender/templates/initscripts/pdfrender.systemd.erb 
b/modules/pdfrender/templates/initscripts/pdfrender.systemd.erb
index 598fdb4..a2c3f22 100644
--- a/modules/pdfrender/templates/initscripts/pdfrender.systemd.erb
+++ b/modules/pdfrender/templates/initscripts/pdfrender.systemd.erb
@@ -7,6 +7,7 @@
 [Service]
 # up ulimit -n a bit
 LimitNOFILE=1
+MemoryLimit=2G
 User=pdfrender
 Group=pdfrender
 Environment="NODE_PATH=/srv/deployment/electron-render/deploy/node_modules"

-- 
To view, visit https://gerrit.wikimedia.org/r/35
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If199a0271b7d36e26a3dd90e9c130fd71eb78945
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Update sampled logging levels to debug

2017-05-26 Thread GWicke (Code Review)

GWicke has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/355801 )

Change subject: Update sampled logging levels to debug
..

Update sampled logging levels to debug

https://github.com/wikimedia/hyperswitch/pull/78 switched the default
logging level for sampled logs from "trace" to "debug". This patch
updates the configuration to match, which should restore sampled logs in
production.

Change-Id: I8cf34d559e553349a18ec8f89fdafaf13f8a952d
---
M scap/templates/config.yaml.j2
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/01/355801/1

diff --git a/scap/templates/config.yaml.j2 b/scap/templates/config.yaml.j2
index 35a707e..b9a1949 100644
--- a/scap/templates/config.yaml.j2
+++ b/scap/templates/config.yaml.j2
@@ -26,9 +26,9 @@
   type: file
   sampled_levels:
 # Target is about 1 request logged per minute @1000 req/s
-trace/request: 0.5
+debug/request: 0.5
 # Not sure what the rate of these is, so start conservatively.
-trace/request/slow: 0.001
+debug/request/slow: 0.001
 
 # Statsd metrics reporter
 metrics:

-- 
To view, visit https://gerrit.wikimedia.org/r/355801
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8cf34d559e553349a18ec8f89fdafaf13f8a952d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...Math[master]: Better error handling for math render errors

2017-05-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/353056 )

Change subject: Better error handling for math render errors
..

Better error handling for math render errors

So far, rare render errors caused by blips in the network or the mathoid
service produce a big red warning in the page, and cause a tracking
category to be added. The rendered content along with the error message
is cached normally, which means that temporary errors are persisted
indefinitely.

As a consequence, editors need to periodically perform null edits to
remove temporary render errors from caches. T136812 documents a history
of this.

This patch aims to improve error handling through two changes:

1) Limit caching of rendered content to 10 minutes. As a consequence,
   temporary render errors should clear themselves up fairly quickly,
   eliminating the need for manual follow-up. The 10 minute timeout was
   chose as a compromise between relatively timely cleanup, and limited
   load from re-renders after math trouble.

2) Track render errors in a dedicated category. This avoids mixing
   syntax errors with temporary render errors, which should make cleanup
   efforts a lot simpler.

Change-Id: If7537ed9117fb736a09f4396b7d55d774ce9674f
---
M Math.hooks.php
1 file changed, 5 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Math 
refs/changes/56/353056/1

diff --git a/Math.hooks.php b/Math.hooks.php
index 2950b66..223b80e 100644
--- a/Math.hooks.php
+++ b/Math.hooks.php
@@ -229,7 +229,11 @@
} else {
LoggerFactory::getInstance( 'Math' )->warning(
"Rendering failed. Printing error message." );
-   $parser->addTrackingCategory( 
'math-tracking-category-error' );
+   // Set a short parser cache time (10 minutes) after 
encountering
+   // render issues, but not syntax issues.
+   $parser->getOutput()->updateCacheExpiry( 600 );
+   // Add a tracking category specialized on render errors.
+   $parser->addTrackingCategory( 
'math-tracking-category-render-error' );
return $renderer->getLastError();
}
Hooks::run( 'MathFormulaPostRender',

-- 
To view, visit https://gerrit.wikimedia.org/r/353056
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If7537ed9117fb736a09f4396b7d55d774ce9674f
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Math
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: WIP: Add cache-control option that allows for short term cli...

2017-03-31 Thread GWicke (Code Review)

GWicke has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/345877 )

Change subject: WIP: Add cache-control option that allows for short term client 
caching
..

WIP: Add cache-control option that allows for short term client caching

Some end points can allow for some short term client side caching, which
helps to reduce repeat requests during active use.

Bug: T161284
Change-Id: I2287249ac6a8c22fe904b62ea9f1b591db7497f5
---
M modules/restbase/templates/config.labs.yaml.erb
M modules/restbase/templates/config.yaml.erb
2 files changed, 6 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/77/345877/1

diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index 065e2c3..a12eccb 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -57,6 +57,9 @@
   cache_control: s-maxage=86400, max-age=86400
 # Cache headers for purged end points.
 purged_cache_control: s-maxage=1209600, max-age=0, must-revalidate
+# Cache headers for purged end points, which allows for some short
+# term client side caching.
+purged_some_client_cache_control: s-maxage=1209600, max-age=120
 pdf:
   cache_control: s-maxage=600, max-age=600
   uri: <%= @pdfrender_uri %>
diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 2c81af1..f74803e 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -79,6 +79,9 @@
   cache_control: s-maxage=86400, max-age=3600
 # Cache headers for purged end points.
 purged_cache_control: s-maxage=1209600, max-age=0, must-revalidate
+# Cache headers for purged end points, which allows for some short
+# term client side caching.
+purged_some_client_cache_control: s-maxage=1209600, max-age=120
 skip_updates: <%= @skip_updates %>
 pdf:
   cache_control: s-maxage=600, max-age=600

-- 
To view, visit https://gerrit.wikimedia.org/r/345877
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2287249ac6a8c22fe904b62ea9f1b591db7497f5
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Update access log sampling to match new hyperswitch levels

2017-03-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/342251 )

Change subject: Update access log sampling to match new hyperswitch levels
..

Update access log sampling to match new hyperswitch levels

Follow-up change from https://github.com/wikimedia/hyperswitch/pull/78.
Should be merged & deployed at the same time.

Change-Id: Ia129a9a0083368684b32c5eaffb2c01ec14962f5
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/51/342251/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index ada8eb1..e00a25a 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -940,9 +940,9 @@
   level: <%= @logging_level %>
   sampled_levels:
 # Target is about 1 request logged per minute @1000 req/s
-trace/request: 0.5
+info/request: 0.5
 # Not sure what the rate of these is, so start conservatively.
-trace/request/slow: 0.001
+info/request/slow: 0.001
   streams:
   # XXX: Use gelf-stream -> logstash
   - type: gelf

-- 
To view, visit https://gerrit.wikimedia.org/r/342251
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia129a9a0083368684b32c5eaffb2c01ec14962f5
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Delay service shut-down to work around xpra race

2017-03-08 Thread GWicke (Code Review)

GWicke has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/341833 )

Change subject: Delay service shut-down to work around xpra race
..

Delay service shut-down to work around xpra race

Bug: T159922
Change-Id: I0a73a44dbd48c615efd63f1db0ac5da1b5f56e3c
---
M modules/pdfrender/templates/initscripts/pdfrender.systemd.erb
1 file changed, 3 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/33/341833/1

diff --git a/modules/pdfrender/templates/initscripts/pdfrender.systemd.erb 
b/modules/pdfrender/templates/initscripts/pdfrender.systemd.erb
index fefc0c3..598fdb4 100644
--- a/modules/pdfrender/templates/initscripts/pdfrender.systemd.erb
+++ b/modules/pdfrender/templates/initscripts/pdfrender.systemd.erb
@@ -23,6 +23,9 @@
 TimeoutStopSec=60
 WorkingDirectory=/srv/deployment/electron-render/deploy
 ExecStart=/usr/bin/firejail --profile=/etc/firejail/pdfrender.profile 
/usr/bin/nodejs 
/srv/deployment/electron-render/deploy/src/bin/electron-render-service.js
+# Delay restart for 10s to work around xpra shutdown race
+# See https://phabricator.wikimedia.org/T159922.
+ExecStopPost=/bin/sleep 10
 
 [Install]
 WantedBy=multi-user.target

-- 
To view, visit https://gerrit.wikimedia.org/r/341833
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0a73a44dbd48c615efd63f1db0ac5da1b5f56e3c
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Whitelist /home/pdfrender/.config in firejail profile

2016-12-02 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/324976

Change subject: Whitelist /home/pdfrender/.config in firejail profile
..

Whitelist /home/pdfrender/.config in firejail profile

The firejail default profile does not grant permission to read a user's
home directory, so we need to explicitly whitelist the .config directory
containing fonts.conf to enable hinting.

Change-Id: I8fb502f6a10ef666bb006d495b4335472c53fb02
---
M modules/pdfrender/files/firejail.profile
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/76/324976/1

diff --git a/modules/pdfrender/files/firejail.profile 
b/modules/pdfrender/files/firejail.profile
index a9e3c46..eee8468 100644
--- a/modules/pdfrender/files/firejail.profile
+++ b/modules/pdfrender/files/firejail.profile
@@ -8,6 +8,7 @@
 include /etc/firejail/default.profile
 
 blacklist /root
+whitelist /home/pdfrender/.config
 
 # Electron uses xvfb as the X11 engine by default, which
 # requires setuid root; that is not acceptable for firejail

-- 
To view, visit https://gerrit.wikimedia.org/r/324976
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8fb502f6a10ef666bb006d495b4335472c53fb02
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Add fontconfig file for the pdf render service

2016-12-01 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/324747

Change subject: Add fontconfig file for the pdf render service
..

Add fontconfig file for the pdf render service

Enable font hinting by installing the default electron-render-service
.fonts.conf [1] in the service user's home directory. Alternatively, we
could mess with the global configuration, but this seemed to be the
simpler approach, and is confirmed to work in labs.

[1]: https://github.com/msokk/electron-render-service/blob/master/.fonts.conf

Change-Id: I35789b89d50260ca79aa97576836f4b35463a776
---
A modules/pdfrender/files/fonts.conf
M modules/pdfrender/manifests/init.pp
2 files changed, 38 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/47/324747/1

diff --git a/modules/pdfrender/files/fonts.conf 
b/modules/pdfrender/files/fonts.conf
new file mode 100644
index 000..fdc875b
--- /dev/null
+++ b/modules/pdfrender/files/fonts.conf
@@ -0,0 +1,29 @@
+
+
+
+ 
+  
+   rgb
+  
+ 
+ 
+  
+   true
+  
+ 
+ 
+  
+   hintslight
+  
+ 
+ 
+  
+   true
+  
+ 
+  
+
+  lcddefault
+
+  
+
diff --git a/modules/pdfrender/manifests/init.pp 
b/modules/pdfrender/manifests/init.pp
index a1a48ca..afefbc9 100644
--- a/modules/pdfrender/manifests/init.pp
+++ b/modules/pdfrender/manifests/init.pp
@@ -100,6 +100,15 @@
 before => Base::Service_unit['pdfrender'],
 }
 
+file { '${home_dir}/.fonts.conf':
+ensure => present,
+owner  => 'pdfrender',
+group  => 'pdfrender',
+mode   => '0444',
+source => 'puppet:///modules/pdfrender/fonts.conf',
+before => Base::Service_unit['pdfrender'],
+}
+
 systemd::syslog { 'pdfrender':
 readable_by => 'all',
 base_dir=> $::service::configuration::log_dir,

-- 
To view, visit https://gerrit.wikimedia.org/r/324747
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I35789b89d50260ca79aa97576836f4b35463a776
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...graphoid[master]: Test node 6 & latest

2016-10-31 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/319009

Change subject: Test node 6 & latest
..

Test node 6 & latest

Change-Id: I49517351e8cd14a529b4236e982174f33aa6debe
---
M .travis.yml
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/graphoid 
refs/changes/09/319009/1

diff --git a/.travis.yml b/.travis.yml
index f33adc7..34ca0bf 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -6,4 +6,5 @@
   - "0.10"
   - "0.12"
   - "4"
-  - "5"
+  - "6"
+  - "node"

-- 
To view, visit https://gerrit.wikimedia.org/r/319009
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I49517351e8cd14a529b4236e982174f33aa6debe
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/graphoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...citoid[master]: Test node 6 & latest

2016-10-31 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/319008

Change subject: Test node 6 & latest
..

Test node 6 & latest

Change-Id: I982f898f9db7f33e1f4551fc18122b8bbd4421cb
---
M .travis.yml
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/citoid 
refs/changes/08/319008/1

diff --git a/.travis.yml b/.travis.yml
index f33adc7..34ca0bf 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -6,4 +6,5 @@
   - "0.10"
   - "0.12"
   - "4"
-  - "5"
+  - "6"
+  - "node"

-- 
To view, visit https://gerrit.wikimedia.org/r/319008
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I982f898f9db7f33e1f4551fc18122b8bbd4421cb
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/citoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...parsoid[master]: Test with node 4 & 6

2016-10-31 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/318999

Change subject: Test with node 4 & 6
..

Test with node 4 & 6

Change-Id: I29ce0c8152189723eec20d63011877a0822ac098
---
M .travis.yml
1 file changed, 4 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/99/318999/1

diff --git a/.travis.yml b/.travis.yml
index 563d7a0..461fd8f 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -3,10 +3,10 @@
 node_js:
 # Keep this around until we deprecate and remove it.
 - "0.10.25"
-# Current production value.
-- "4.4.6"
-# Production, next
-- "4.6.0"
+# Production
+- "4"
+# Next LTS
+- "6"
 # From the nvm docs, "this installs the latest version of node".
 - "node"
 

-- 
To view, visit https://gerrit.wikimedia.org/r/318999
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I29ce0c8152189723eec20d63011877a0822ac098
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...mobileapps[master]: Test with node 6 & latest

2016-10-31 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/318996

Change subject: Test with node 6 & latest
..

Test with node 6 & latest

Change-Id: Ie4b651f5ce082cd3d53c6ddcd342f4c5d456e6a6
---
M .travis.yml
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/mobileapps 
refs/changes/96/318996/1

diff --git a/.travis.yml b/.travis.yml
index f33adc7..fa7f265 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -6,4 +6,5 @@
   - "0.10"
   - "0.12"
   - "4"
-  - "5"
+  - "6"
+  = "node"

-- 
To view, visit https://gerrit.wikimedia.org/r/318996
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie4b651f5ce082cd3d53c6ddcd342f4c5d456e6a6
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/mobileapps
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] analytics/aqs[master]: Travis test with node 4, 6 & latest.

2016-10-31 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/318995

Change subject: Travis test with node 4, 6 & latest.
..

Travis test with node 4, 6 & latest.

Change-Id: I6865e689c2821536ab442b8ebe642c791128f3e6
---
M .travis.yml
1 file changed, 3 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/analytics/aqs 
refs/changes/95/318995/1

diff --git a/.travis.yml b/.travis.yml
index 61cc69e..8e433f9 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -1,9 +1,8 @@
 language: node_js
 node_js:
-- "0.10"
-- "0.12"
-- "4.2"
-- "5"
+- "4"
+- "6"
+- "node"
 
 sudo: false
 

-- 
To view, visit https://gerrit.wikimedia.org/r/318995
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6865e689c2821536ab442b8ebe642c791128f3e6
Gerrit-PatchSet: 1
Gerrit-Project: analytics/aqs
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] labs...wikibugs2[master]: Update wikimedia-services mapping

2016-10-13 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/315830

Change subject: Update wikimedia-services mapping
..

Update wikimedia-services mapping

- Try to match services milestones.
- Add service-runner and cassandra projects.

Change-Id: Id6f977d924406a4eb7037a4a119f08f595961203
---
M channels.yaml
1 file changed, 4 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/labs/tools/wikibugs2 
refs/changes/30/315830/1

diff --git a/channels.yaml b/channels.yaml
index 2ed5848..5e1a42a 100644
--- a/channels.yaml
+++ b/channels.yaml
@@ -167,8 +167,10 @@
 - MediaWiki-extensions-ParserFunctions
 
 "#wikimedia-services":
-- restbase(-.*)?
-- services(-.*)?
+- restbase.*
+- services.*
+- service-runner
+- cassandra
 - blocked-on-services
 
 "#wikimedia-multimedia":

-- 
To view, visit https://gerrit.wikimedia.org/r/315830
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id6f977d924406a4eb7037a4a119f08f595961203
Gerrit-PatchSet: 1
Gerrit-Project: labs/tools/wikibugs2
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Add security header filters

2016-09-08 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/309486

Change subject: Add security header filters
..

Add security header filters

Update for the deployment of https://github.com/wikimedia/restbase/pull/665.

Change-Id: I297623e6c3b990122d28dae0a33e3ad6a4b5ad46
---
M modules/restbase/templates/config.labs.yaml.erb
M modules/restbase/templates/config.yaml.erb
2 files changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/86/309486/1

diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index 766b4ae..45f188e 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -62,6 +62,8 @@
 root_spec: _spec
   title: "The RESTBase root"
   # Some more general RESTBase info
+  x-request-filters:
+- path: lib/security_response_header_filter.js
   x-sub-request-filters:
 - type: default
   name: http
diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index d350314..30ae58f 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -91,6 +91,8 @@
 root_spec: _spec
   title: "The RESTBase root"
   # Some more general RESTBase info
+  x-request-filters:
+- path: lib/security_response_header_filter.js
   x-sub-request-filters:
 - type: default
   name: http

-- 
To view, visit https://gerrit.wikimedia.org/r/309486
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I297623e6c3b990122d28dae0a33e3ad6a4b5ad46
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...OfflineContentGenerator[master]: Add a metric for number of items in collections

2016-08-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/304043

Change subject: Add a metric for number of items in collections
..

Add a metric for number of items in collections

For planning purposes, we would like to know how many requests are for a
single page vs. multi-page collections. To this end, it seems useful to record
the size of requested collections in a metric.

This patch adds a timing metric for this purpose, which gives us percentiles
in graphite.

Change-Id: I0f91778f0d71bbb66b719b6b3e5dd19a4de61962
---
M lib/threads/frontend.js
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Collection/OfflineContentGenerator
 refs/changes/43/304043/1

diff --git a/lib/threads/frontend.js b/lib/threads/frontend.js
index 25894e0..b7426f6 100644
--- a/lib/threads/frontend.js
+++ b/lib/threads/frontend.js
@@ -793,6 +793,8 @@
}
});
};
+   // Use a timing metric, so that we get percentiles.
+   statsd.timing('frontend.requests.createCollectionId.itemCount', 
metabookObj.items.length);
updateChapter(metabookObj.items);
 
// XXX in the future we might separately cache the bundle.zip and

-- 
To view, visit https://gerrit.wikimedia.org/r/304043
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0f91778f0d71bbb66b719b6b3e5dd19a4de61962
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Collection/OfflineContentGenerator
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Forward Accept also for /api/rest_v1/?doc

2016-08-09 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/303825

Change subject: Forward Accept also for /api/rest_v1/?doc
..

Forward Accept also for /api/rest_v1/?doc

Follow-up to I2fbcbd97fb1335061519dbee1e7222ae8e5c8617.

Change-Id: I4aaa09a839b4a08c392e8fa70f91a3c4a7da10a6
---
M templates/varnish/text-common.inc.vcl.erb
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/25/303825/1

diff --git a/templates/varnish/text-common.inc.vcl.erb 
b/templates/varnish/text-common.inc.vcl.erb
index 67eb71d..c556cf3 100644
--- a/templates/varnish/text-common.inc.vcl.erb
+++ b/templates/varnish/text-common.inc.vcl.erb
@@ -68,7 +68,7 @@
 
// Normalize Accept headers for the REST API: Ignore unless a profile is
// specified or this is a request for the root REST URI
-   if (req.url ~ "^/api/rest_v1/?$" && req.http.Accept ~ "text/html") {
+   if (req.url ~ "^/api/rest_v1/(\?doc)?$" && req.http.Accept ~ 
"text/html") {
set req.http.Accept = "text/html";
} else if (req.url ~ "^/api/rest_v1/" && req.http.Accept !~ 
{"profile="https://www.mediawiki.org/wiki/Specs/"}) {
unset req.http.Accept;

-- 
To view, visit https://gerrit.wikimedia.org/r/303825
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4aaa09a839b4a08c392e8fa70f91a3c4a7da10a6
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Service::node: Capture stdout and stderr in journal

2016-07-26 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/301309

Change subject: Service::node: Capture stdout and stderr in journal
..

Service::node: Capture stdout and stderr in journal

Fatal errors frequently do not make it into regular service logs. During
startup, other logging infrastructure might not be available yet, and sudden
fatals don't necessarily leave time to actually send out logs before the
process exits.

Production services generally do not log to stdout / stderr (they use logstash
or separate log files), so it should be safe to capture such output to the
systemd journal by default.

Bug: T136957
Change-Id: Ib038cb6bd9312c17875a60c476d846efd2f89c6f
---
M modules/service/templates/initscripts/node.systemd.erb
1 file changed, 5 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/09/301309/1

diff --git a/modules/service/templates/initscripts/node.systemd.erb 
b/modules/service/templates/initscripts/node.systemd.erb
index f9de25e..fb98071 100644
--- a/modules/service/templates/initscripts/node.systemd.erb
+++ b/modules/service/templates/initscripts/node.systemd.erb
@@ -20,5 +20,10 @@
 WorkingDirectory=/srv/deployment/<%= @repo %>
 ExecStart=/usr/bin/firejail --blacklist=/root --blacklist=/home --caps 
--seccomp /usr/bin/nodejs <%= @starter_script %> -c /etc/<%= @title 
%>/config.yaml
 
+# Always redirect stdout and stderr to the journal. Production services only
+# log fatal errors to stderr / stdout, and use logstash (or log files) for
+# regular app logs. StandardError is piped to stdout by systemd's defaults.
+StandardOutput=journal
+
 [Install]
 WantedBy=multi-user.target

-- 
To view, visit https://gerrit.wikimedia.org/r/301309
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib038cb6bd9312c17875a60c476d846efd2f89c6f
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Lower trickle_fsync_interval to 8mb - change (operations/puppet)

2016-07-20 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/300100

Change subject: Lower trickle_fsync_interval to 8mb
..

Lower trickle_fsync_interval to 8mb

In production, we still see relatively bursty IO, with no or few writes
followed by a burst every five seconds or so. This causes short spikes of
iowait up into the dozens of percent, which in turn do impact p99 read 
latencies.

Our current trickle fsync interval is 30mb, which is rather large. I think
it's worth lowering this significantly, perhaps to 8mb. All of those values
are significantly above typical SSD erase block sizes of 256k.

Bug: T140825

Change-Id: I9d6d2517f9ea90f99cd52c94483064ced9074324
---
M modules/cassandra/templates/cassandra.yaml-2.2.erb
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/00/300100/1

diff --git a/modules/cassandra/templates/cassandra.yaml-2.2.erb 
b/modules/cassandra/templates/cassandra.yaml-2.2.erb
index 4f4d153..f05c5b7 100644
--- a/modules/cassandra/templates/cassandra.yaml-2.2.erb
+++ b/modules/cassandra/templates/cassandra.yaml-2.2.erb
@@ -403,7 +403,7 @@
 # impacting read latencies. Almost always a good idea on SSDs; not
 # necessarily on platters.
 trickle_fsync: true
-trickle_fsync_interval_in_kb: 30240
+trickle_fsync_interval_in_kb: 8192
 
 # TCP port, for commands and data
 # For security reasons, you should not expose this port to the internet.  
Firewall it if needed.

-- 
To view, visit https://gerrit.wikimedia.org/r/300100
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9d6d2517f9ea90f99cd52c94483064ced9074324
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] For discussion: Reduce purge volume by moving dependent purg... - change (mediawiki/core)

2016-06-17 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/295027

Change subject: For discussion: Reduce purge volume by moving dependent purges 
to RefreshLinksJob
..

For discussion: Reduce purge volume by moving dependent purges to 
RefreshLinksJob

On edit, we currently create

a) a HTMLCacheUpdateJob, which
  - updates page_touched for all pages transcluded by a page, and
  - performs a CDN purge of all those pages.
b) a RefreshLinksJob, which will re-render the same set of pages, but won't
  purge them.

RefreshLinksJob is significantly more expensive than HTMLCacheUpdateJob, and
takes a while when a template used in millions of pages was edited.
HTMLCacheUpdateJob on the other hand only performs relatively cheap database
queries, and quickly sends out a lot of CDN purges.

The chief advantage of this scheme is timely CDN purges, which ensures that
anonymous users quickly pick up template edits across the site. Disadvantages
are:

- The quick processing of CDN purges can result in bursts of very high purge
  rates, and minimizes the chances of coalescing purges from several quick
  edits into a single CDN purge.
- High-volume anonymous traffic will trigger parser cache misses after the
  HTMLCacheUpdateJob has executed. This leads to higher latency for users, and
  can create spikes in the load of app servers, databases, memcached etc.

This patch addresses these issues by moving CDN purges from HTMLCacheUpdateJob
to RefreshLinksJob. As a consequence, the following changes are expected:

- CDN purges should be less bursty, as RefreshLinksJob processing rates are
  more limited.
- Multiple edits to the same popular template should result in only a single
  CDN purge for the vast majority of pages using the template, as subsequent
  page_touched increments will abort earlier refreshlinks jobs & purges. It is
  expected that this should reduce the overall rate of CDN purges
  signficantly.
- By purging the CDN only after the page cache has been updated, anonymous
  traffic is no longer going to hit parser cache misses with the associated
  latency increase, and won't cause a spike in load on the infrastructure from
  a high rate of re-renders.

However, the downside is clearly that the purging of dependent pages is going
to be delayed, in line with the pace of RefreshLinksJob processing. This will
only affect anonymous users, as authenticated users will still trigger
immediate re-renders based on page_touched. It will also not affect the edited
pages themselves, as those are still purged immediately.

I believe that considering the performance and stability benefits of this
change, this is a reasonable trade-off to make. However, this is a judgment
call, which is why I am posting this patch for discussion.

Change-Id: Idb2867e2d90b11aa1bf2f249058b24c0c0a92036
---
M includes/jobqueue/jobs/HTMLCacheUpdateJob.php
M includes/jobqueue/jobs/RefreshLinksJob.php
2 files changed, 9 insertions(+), 18 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/27/295027/1

diff --git a/includes/jobqueue/jobs/HTMLCacheUpdateJob.php 
b/includes/jobqueue/jobs/HTMLCacheUpdateJob.php
index 7acbdf2..14cd38b 100644
--- a/includes/jobqueue/jobs/HTMLCacheUpdateJob.php
+++ b/includes/jobqueue/jobs/HTMLCacheUpdateJob.php
@@ -131,24 +131,6 @@
__METHOD__
);
}
-   // Get the list of affected pages (races only mean something 
else did the purge)
-   $titleArray = TitleArray::newFromResult( $dbw->select(
-   'page',
-   [ 'page_namespace', 'page_title' ],
-   [ 'page_id' => $pageIds, 'page_touched' => 
$dbw->timestamp( $touchTimestamp ) ],
-   __METHOD__
-   ) );
-
-   // Update CDN
-   $u = CdnCacheUpdate::newFromTitles( $titleArray );
-   $u->doUpdate();
-
-   // Update file cache
-   if ( $wgUseFileCache ) {
-   foreach ( $titleArray as $title ) {
-   HTMLFileCache::clearFileCache( $title );
-   }
-   }
}
 
public function workItemCount() {
diff --git a/includes/jobqueue/jobs/RefreshLinksJob.php 
b/includes/jobqueue/jobs/RefreshLinksJob.php
index 8870569..495d648 100644
--- a/includes/jobqueue/jobs/RefreshLinksJob.php
+++ b/includes/jobqueue/jobs/RefreshLinksJob.php
@@ -264,6 +264,15 @@
 
InfoAction::invalidateCache( $title );
 
+   // Update CDN
+   $u = CdnCacheUpdate::newSimplePurge( $title );
+   $u->doUpdate();
+
+   // Update file cache
+   if ( $wgUseFileCache ) {
+   HTMLFileCache::clearFileCache( $title );
+   }
+
return true;
}
 

--

[MediaWiki-commits] [Gerrit] WIP: logstash_checker script for canary deploys - change (operations/puppet)

2016-06-02 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/292505

Change subject: WIP: logstash_checker script for canary deploys
..

WIP: logstash_checker script for canary deploys

This is a work-in progress checker script for checking logstash error rates
before/after a deploy.

Bug: T110068
Change-Id: I1a900ee1d7eadc4689e14306a2fc72ad2c138a28
---
A modules/service/files/logstash_checker.py
1 file changed, 217 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/05/292505/1

diff --git a/modules/service/files/logstash_checker.py 
b/modules/service/files/logstash_checker.py
new file mode 100755
index 000..05b2f73
--- /dev/null
+++ b/modules/service/files/logstash_checker.py
@@ -0,0 +1,217 @@
+#!/usr/bin/python
+
+"""
+Basic logstash error rate checker
+
+Theory of operation:
+- fetch histogram of error / fatals from logstash for the last ~10 minutes
+- calculate the mean rates before/after a time  seconds in the past
+- if the `after` rate is more than  times the before rate,
+  return an error; else, exit with 0.
+"""
+
+
+import sys
+reload(sys)
+sys.setdefaultencoding('utf-8')
+
+try:
+import urlparse
+except ImportError:
+import urllib.parse as urlparse
+import json
+import urllib3
+import sys
+import argparse
+import re
+import urllib
+import time
+from collections import namedtuple
+
+
+class CheckServiceError(Exception):
+
+"""
+Generic Exception used as a catchall
+"""
+pass
+
+
+def fetch_url(client, url, **kw):
+"""
+Standalone function to fetch an url.
+
+Args:
+client (urllib3.Poolmanager):
+ The HTTP client we want to use
+url (str): The URL to fetch
+
+kw: any keyword arguments we want to pass to
+urllib3.request.RequestMethods.request
+"""
+if 'method' in kw:
+method = kw['method'].upper()
+del kw['method']
+else:
+method = 'GET'
+try:
+if method == 'GET':
+return client.request(
+method,
+url,
+**kw
+)
+elif method == 'POST':
+try:
+headers = kw.get('headers', {})
+content_type = headers.get('content-type', '')
+except:
+content_type = ''
+
+# Handle json-encoded requests
+if content_type.lower() == 'application/json':
+kw['body'] = json.dumps(kw['body'])
+return client.urlopen(
+method,
+url,
+**kw
+)
+
+return client.request_encode_body(
+method,
+url,
+encode_multipart=False,
+**kw
+)
+except urllib3.exceptions.SSLError:
+raise CheckServiceError("Invalid certificate")
+except (urllib3.exceptions.ConnectTimeoutError,
+urllib3.exceptions.TimeoutError,
+# urllib3.exceptions.ConnectionError, # commented out until we can
+# remove trusty (aka urllib3 1.7.1) support
+urllib3.exceptions.ReadTimeoutError):
+raise CheckServiceError("Timeout on connection while "
+"downloading {}".format(url))
+except Exception as e:
+raise CheckServiceError("Generic connection error: {}".format(e))
+
+
+class CheckService(object):
+
+"""
+Shell class for checking services
+"""
+nagios_codes = ['OK', 'WARNING', 'CRITICAL']
+spec_url = '/?spec'
+default_response = {'status': 200}
+_supported_methods = ['get', 'post']
+
+def __init__(self, args):
+"""
+Initialize the checker
+
+Args:
+host_ip (str): The host ipv4 address (also works with a hostname)
+
+base_url (str): The base url the service expects to respond from
+
+timeout (int): Number of seconds to wait for each request
+"""
+self.host = args.host
+self.service_name = args.service_name
+self.delay = args.delay
+self.threshold = 2
+self.logstash_query_url = 
'https://logstash.wikimedia.org/logstash-2016.06.02/_search'
+#args.logstash_query_url
+
+def _logstashQuery(self):
+return {"facets":{"0":{
+"date_histogram":{"field":"@timestamp","interval":"10s"},
+"global":True,
+"facet_filter":{"fquery":{"query":{
+"filtered":{"query":{"query_string":{"query":"*"}},
+"filter":{"bool":{
+"must":[
+
{"range":{"@timestamp":{"from":"now-10m","to":"now"}}},
+{"terms":{"_type":["mediawiki"]}},
+
{"fquery":{"query":{"query_string":{"query":"host:(\""

[MediaWiki-commits] [Gerrit] Update restbase to f05b66f - change (mediawiki...deploy)

2016-06-01 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/292299

Change subject: Update restbase to f05b66f
..

Update restbase to f05b66f

List of changes:
xxx Update node module dependencies

Change-Id: I68b658ec13027c535de56712c3779721079043f6
---
M node_modules/content-type/package.json
M node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
M node_modules/hyperswitch/node_modules/swagger-ui/package.json
M node_modules/json-stable-stringify/package.json
M 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/core-util-is/package.json
M 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/string_decoder/package.json
M node_modules/restbase-mod-table-cassandra/lib/index.js
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/json-stable-stringify/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/routeswitch/node_modules/async/package.json
M node_modules/restbase-mod-table-cassandra/package.json
M 
node_modules/service-runner/node_modules/limitation/node_modules/kad/package.json
M 
node_modules/service-runner/node_modules/limitation/node_modules/readable-stream/node_modules/core-util-is/package.json
M 
node_modules/service-runner/node_modules/limitation/node_modules/readable-stream/node_modules/inherits/package.json
M 
node_modules/service-runner/node_modules/limitation/node_modules/readable-stream/node_modules/string_decoder/package.json
M 
node_modules/service-runner/node_modules/yargs/node_modules/find-up/package.json
16 files changed, 32 insertions(+), 28 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/99/292299/1

diff --git a/node_modules/content-type/package.json 
b/node_modules/content-type/package.json
index e178aa2..1d4b257 100644
--- a/node_modules/content-type/package.json
+++ b/node_modules/content-type/package.json
@@ -44,7 +44,7 @@
   },
   "homepage": "https://github.com/jshttp/content-type#readme;,
   "_id": "content-type@1.0.1",
-  "_shasum": "19cf3b5f4f4218590a5f89ac87ba8d94465fbad4",
+  "_shasum": "af353d5991784749680b7e3f14c7c5caba9e83ef",
   "_from": "git+https://github.com/wikimedia/content-type.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/content-type.git#47b2632d0a2ee79a7d67268e2f6621becd95d05b;
 }
diff --git 
a/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json 
b/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
index 11abd1a..ca71f8c 100644
--- a/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
+++ b/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
@@ -50,7 +50,7 @@
 "url": "https://github.com/substack/json-stable-stringify/issues;
   },
   "_id": "json-stable-stringify@1.0.0",
-  "_shasum": "557e6749ba3ef6e070f845044a26c4d3ce3bace6",
+  "_shasum": "58fdcf5538bee09def8d0d43cc7bd8bb7ebb5aa0",
   "_from": "git+https://github.com/wikimedia/json-stable-stringify.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/json-stable-stringify.git#c05178e4dfc3834017f017732934c1c52ba3dd8f;
 }
diff --git a/node_modules/hyperswitch/node_modules/swagger-ui/package.json 
b/node_modules/hyperswitch/node_modules/swagger-ui/package.json
index 441a066..f41f4de 100644
--- a/node_modules/hyperswitch/node_modules/swagger-ui/package.json
+++ b/node_modules/hyperswitch/node_modules/swagger-ui/package.json
@@ -60,7 +60,7 @@
 "url": "https://github.com/swagger-api/swagger-ui/issues;
   },
   "_id": "swagger-ui@2.1.4",
-  "_shasum": "22fffd49f3722b827ca1678efd9a0dac16c6023b",
+  "_shasum": "af0a93e9cb5da25ef020ade4ac0f6e46f7246807",
   "_from": "git+https://github.com/wikimedia/swagger-ui.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/swagger-ui.git#7bef30a93a3b81c55cb95d3ce9e0f9f53d5d8262;
 }
diff --git a/node_modules/json-stable-stringify/package.json 
b/node_modules/json-stable-stringify/package.json
index 2d5905a..1831a98 100644
--- a/node_modules/json-stable-stringify/package.json
+++ b/node_modules/json-stable-stringify/package.json
@@ -50,7 +50,7 @@
 "url": "https://github.com/substack/json-stable-stringify/issues;
   },
   "_id": "json-stable-stringify@1.0.0",
-  "_shasum": "9144642a745e113e626d46363cb14bc01cd6bce2",
+  "_shasum": "d662c1fea6cc45c576d5789a1910e021d9253f12",
   "_from": "git+https://github.com/wikimedia/json-stable-stringify.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/json-stable-stringify.git#c05178e4dfc3834017f017732934c1c52ba3dd8f;
 }
diff --git

[MediaWiki-commits] [Gerrit] Update restbase to f05b66f - change (mediawiki...deploy)

2016-06-01 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/292292

Change subject: Update restbase to f05b66f
..

Update restbase to f05b66f

List of changes:
xxx Update node module dependencies

Change-Id: I75495f9d215d7c5eedf7cd35a44b59505b6b5667
---
M node_modules/content-type/package.json
M node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
M node_modules/hyperswitch/node_modules/swagger-ui/package.json
M node_modules/json-stable-stringify/package.json
M 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/core-util-is/package.json
M 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/string_decoder/package.json
M node_modules/restbase-mod-table-cassandra/lib/index.js
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/json-stable-stringify/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/routeswitch/node_modules/async/package.json
M node_modules/restbase-mod-table-cassandra/package.json
A node_modules/service-runner/node_modules/hot-shots/CHANGES.md~
M 
node_modules/service-runner/node_modules/limitation/node_modules/kad/package.json
M 
node_modules/service-runner/node_modules/limitation/node_modules/readable-stream/node_modules/inherits/package.json
M 
node_modules/service-runner/node_modules/yargs/node_modules/find-up/package.json
A 
node_modules/service-runner/node_modules/yargs/node_modules/read-pkg-up/node_modules/read-pkg/node_modules/normalize-package-data/test/normalize.js~
16 files changed, 319 insertions(+), 27 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/92/292292/1

diff --git a/node_modules/content-type/package.json 
b/node_modules/content-type/package.json
index d39ce79..e178aa2 100644
--- a/node_modules/content-type/package.json
+++ b/node_modules/content-type/package.json
@@ -44,7 +44,7 @@
   },
   "homepage": "https://github.com/jshttp/content-type#readme;,
   "_id": "content-type@1.0.1",
-  "_shasum": "e63477577a966eb13976563ae81c8a6943b1c019",
+  "_shasum": "19cf3b5f4f4218590a5f89ac87ba8d94465fbad4",
   "_from": "git+https://github.com/wikimedia/content-type.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/content-type.git#47b2632d0a2ee79a7d67268e2f6621becd95d05b;
 }
diff --git 
a/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json 
b/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
index 6dd54d9..11abd1a 100644
--- a/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
+++ b/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
@@ -50,7 +50,7 @@
 "url": "https://github.com/substack/json-stable-stringify/issues;
   },
   "_id": "json-stable-stringify@1.0.0",
-  "_shasum": "b40600c78c7d73d00aaa38719a5f8509ec95ad8d",
+  "_shasum": "557e6749ba3ef6e070f845044a26c4d3ce3bace6",
   "_from": "git+https://github.com/wikimedia/json-stable-stringify.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/json-stable-stringify.git#c05178e4dfc3834017f017732934c1c52ba3dd8f;
 }
diff --git a/node_modules/hyperswitch/node_modules/swagger-ui/package.json 
b/node_modules/hyperswitch/node_modules/swagger-ui/package.json
index 32cd9ed..441a066 100644
--- a/node_modules/hyperswitch/node_modules/swagger-ui/package.json
+++ b/node_modules/hyperswitch/node_modules/swagger-ui/package.json
@@ -60,7 +60,7 @@
 "url": "https://github.com/swagger-api/swagger-ui/issues;
   },
   "_id": "swagger-ui@2.1.4",
-  "_shasum": "a0ead9e306316a198b8bbc4e9d489aa0e1c1cc20",
+  "_shasum": "22fffd49f3722b827ca1678efd9a0dac16c6023b",
   "_from": "git+https://github.com/wikimedia/swagger-ui.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/swagger-ui.git#7bef30a93a3b81c55cb95d3ce9e0f9f53d5d8262;
 }
diff --git a/node_modules/json-stable-stringify/package.json 
b/node_modules/json-stable-stringify/package.json
index d823e27..2d5905a 100644
--- a/node_modules/json-stable-stringify/package.json
+++ b/node_modules/json-stable-stringify/package.json
@@ -50,7 +50,7 @@
 "url": "https://github.com/substack/json-stable-stringify/issues;
   },
   "_id": "json-stable-stringify@1.0.0",
-  "_shasum": "db6540972f02b7ee782660ad58c5c36c241f9f34",
+  "_shasum": "9144642a745e113e626d46363cb14bc01cd6bce2",
   "_from": "git+https://github.com/wikimedia/json-stable-stringify.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/json-stable-stringify.git#c05178e4dfc3834017f017732934c1c52ba3dd8f;
 }
diff --git

[MediaWiki-commits] [Gerrit] Use domain-relative URL for client-side RESTBase requests - change (operations/mediawiki-config)

2016-05-27 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/291350

Change subject: Use domain-relative URL for client-side RESTBase requests
..

Use domain-relative URL for client-side RESTBase requests

Some older browsers have bugs in how they expose response headers from CORS
requests. Additionally, cross-origin requests are slower, as they involve a)
an extra DNS lookup, and b) another connection to be set up.

Users on the mobile domains used to send API requests to the main project
domains, making those requests cross-origin. This patch uses an absolute path
instead, which keeps those requests within the mobile domain.

Bug: T135171
Change-Id: Id5d8e726e1126d6148e1d1dc2869c2e0665155c0
---
M wmf-config/CommonSettings.php
1 file changed, 5 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/50/291350/1

diff --git a/wmf-config/CommonSettings.php b/wmf-config/CommonSettings.php
index 32df79a..871aec9 100644
--- a/wmf-config/CommonSettings.php
+++ b/wmf-config/CommonSettings.php
@@ -2133,10 +2133,13 @@
$wgVisualEditorRestbaseURL, 
$wgVisualEditorFullRestbaseURL;
if ( $wmfRealm === 'production' ) {
$wgVisualEditorRestbaseURL = 
"https://$wgServerName/api/rest_v1/page/html/;;
-   $wgVisualEditorFullRestbaseURL = 
"https://$wgServerName/api/rest_;;
+   // Use domain-relative URLs to avoid the need 
for
+   // cross-domain API requests on mobile domains. 
Those are
+   // slower & buggy in some older browsers (see 
T135171).
+   $wgVisualEditorFullRestbaseURL = "/api/rest_";
} elseif ( $wmfRealm === 'labs' ) {
$wgVisualEditorRestbaseURL = 
"//$wgServerName/api/rest_v1/page/html/";
-   $wgVisualEditorFullRestbaseURL = 
"//$wgServerName/api/rest_";
+   $wgVisualEditorFullRestbaseURL = "/api/rest_";
}
};
}

-- 
To view, visit https://gerrit.wikimedia.org/r/291350
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id5d8e726e1126d6148e1d1dc2869c2e0665155c0
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] WIP: exportRestrictions maintenance script - change (mediawiki/core)

2016-05-17 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/289337

Change subject: WIP: exportRestrictions maintenance script
..

WIP: exportRestrictions maintenance script

Bug: T135278
Change-Id: I74aec31c86557b1997e95bf435bca5227f93b0ef
---
A maintenance/exportRestrictions.php
1 file changed, 60 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/37/289337/1

diff --git a/maintenance/exportRestrictions.php 
b/maintenance/exportRestrictions.php
new file mode 100644
index 000..59a9406
--- /dev/null
+++ b/maintenance/exportRestrictions.php
@@ -0,0 +1,60 @@
+http://www.gnu.org/copyleft/gpl.html
+ *
+ * @file
+ * @ingroup Maintenance
+ * @author Gabriel Wicke 
+ */
+
+require_once __DIR__ . '/Maintenance.php';
+
+/**
+ * Maintenance script to dump revision restrictions, one batch at a time.
+ *
+ * @ingroup Maintenance
+ */
+class ExportRevisionRestrictions extends Maintenance {
+
+   public function __construct() {
+   parent::__construct();
+   $this->addDescription( 'Dump revision restrictions, one batch 
at a time.' );
+   $this->addOption( 'start', 'Start from this revision.' );
+   $this->addOption( 'limit', 'Limit the batch size to this 
value.' );
+   }
+
+   public function execute() {
+   $dbr = $this->getDB( DB_SLAVE, 'vslow' );
+   $batchResult = $dbr->select( [ 'revision', 'page' ],
+   [ 'page_title', 'page_id', 'rev_id', 'rev_deleted' ],
+   [ 'rev_id > ' . intval( $this->getOption( 'start', '0' 
) ),
+   'rev_deleted != 0' ],
+   __METHOD__,
+   [ 'ORDER BY' => 'rev_id ASC',
+   'LIMIT' => intval( $this->getOption( 'limit', 
1000 ) ) ],
+   [ 'page_id' => 'rev_page' ] );
+   $rows = [];
+   forEach( $batchResult as $row ) {
+   $rows[] = $row;
+   }
+   $this->output(json_encode($rows));
+   }
+}
+
+$maintClass = "ExportRevisionRestrictions";
+require_once RUN_MAINTENANCE_IF_MAIN;

-- 
To view, visit https://gerrit.wikimedia.org/r/289337
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I74aec31c86557b1997e95bf435bca5227f93b0ef
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Forward x-client-ip & user-agent to AQS - change (operations/puppet)

2016-05-16 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/289092

Change subject: Forward x-client-ip & user-agent to AQS
..

Forward x-client-ip & user-agent to AQS

Change-Id: I08ee687a30fe16e554a023a3a414ba03a0dc3485
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/92/289092/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index d292f8c..c59ecf8 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -95,6 +95,10 @@
 forward_headers: true
   - pattern: <%= @parsoid_uri %>
 forward_headers: true
+  - pattern: <%= @aqs_uri %>
+forward_headers:
+x-client-ip: true
+user-agent: true
   - pattern: /^https?:\/\//
   paths:
 

-- 
To view, visit https://gerrit.wikimedia.org/r/289092
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I08ee687a30fe16e554a023a3a414ba03a0dc3485
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Set conservative retry limits & delays - change (operations/puppet)

2016-05-05 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/287148

Change subject: Set conservative retry limits & delays
..

Set conservative retry limits & delays

Changeprop was previously using the default retry limits & delays of:
- 500ms delay
- up to 5 retries.

This patch changes these values to something more reasonable:
- 60s delay, and
- up to 2 retries.

Bug: T134456
Change-Id: Ifaf44d1562681758e8d44ec7cc9fa1faf99374e5
---
M modules/changeprop/templates/config.yaml.erb
1 file changed, 12 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/48/287148/1

diff --git a/modules/changeprop/templates/config.yaml.erb 
b/modules/changeprop/templates/config.yaml.erb
index cd7d456..6130ae4 100644
--- a/modules/changeprop/templates/config.yaml.erb
+++ b/modules/changeprop/templates/config.yaml.erb
@@ -26,6 +26,12 @@
 templates:
   summary_rerender:
 topic: resource_change
+retry_limit: 2
+# 1 minute
+retry_delay: 6
+retry_on:
+  status:
+- '5xx'
 match:
   meta:
 uri: 
'/^https?:\/\/[^\/]+\/api\/rest_v1\/page\/html\/([^\/]+)$/'
@@ -43,6 +49,12 @@
 
   definition_rerender:
 topic: resource_change
+retry_limit: 2
+# 1 minute
+retry_delay: 6
+retry_on:
+  status:
+- '5xx'
 match:
   meta:
 # These URIs are coming from RESTBase, so we know that 
article titles will be normalized

-- 
To view, visit https://gerrit.wikimedia.org/r/287148
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifaf44d1562681758e8d44ec7cc9fa1faf99374e5
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Print styles: Wrap CSS-generated URLs - change (mediawiki/core)

2016-05-03 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/286772

Change subject: Print styles: Wrap CSS-generated URLs
..

Print styles: Wrap CSS-generated URLs

Chrome does not actually break CSS-generated URL content with word-wrap, which
causes some infoboxes to expand to the entire page.

This patch adds the word-break CSS property, which allows Chrome to break
those URLs at any position. This fixes the rendering of infoboxes on pages
like https://en.wikipedia.org/wiki/San_Francisco.

With basically all PDF generators (including browsers) including clickable
links & many PDFs consumed on-screen it might be worth considering making
print URLs optional at some point, and defaulting to "screen optimized" PDFs.

Change-Id: I8d03032a553643198da6a1dd8b78201a9255fcbb
---
M resources/src/mediawiki.legacy/commonPrint.css
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/72/286772/1

diff --git a/resources/src/mediawiki.legacy/commonPrint.css 
b/resources/src/mediawiki.legacy/commonPrint.css
index c172a7f..efdc022 100644
--- a/resources/src/mediawiki.legacy/commonPrint.css
+++ b/resources/src/mediawiki.legacy/commonPrint.css
@@ -181,6 +181,7 @@
 .mw-body a.external.autonumber:after {
content: " (" attr( href ) ")";
word-wrap: break-word;
+   word-break: break-all;
 }
 
 /* Expand protocol-relative URLs for printing */

-- 
To view, visit https://gerrit.wikimedia.org/r/286772
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8d03032a553643198da6a1dd8b78201a9255fcbb
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Print CSS: Avoid page breaks inside and - change (mediawiki/core)

2016-05-03 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/286721

Change subject: Print CSS: Avoid page breaks inside  and 
..

Print CSS: Avoid page breaks inside  and 

Change-Id: Iee0932286135ea25a7f47fb2e6ddb6e6d669fa7a
---
M resources/src/mediawiki.legacy/commonPrint.css
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/21/286721/1

diff --git a/resources/src/mediawiki.legacy/commonPrint.css 
b/resources/src/mediawiki.legacy/commonPrint.css
index 975ec2a..331be90 100644
--- a/resources/src/mediawiki.legacy/commonPrint.css
+++ b/resources/src/mediawiki.legacy/commonPrint.css
@@ -84,6 +84,8 @@
 }
 
 img,
+figure,
+table,
 .wikitable,
 .thumb {
/* Pagination */

-- 
To view, visit https://gerrit.wikimedia.org/r/286721
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iee0932286135ea25a7f47fb2e6ddb6e6d669fa7a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Normalize REST API Accept headers - change (operations/puppet)

2016-04-01 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/281042

Change subject: Normalize REST API Accept headers
..

Normalize REST API Accept headers

Per T124365, we are going to use `Accept` headers for response format
negotiation. All relevant content-types will contain a profile parameter with
a value pointing to https://www.mediawiki.org/wiki/Specs/, so we can use this
property to normalize the Accept header without fragmenting on random Accept
headers once we enable `Vary: Accept`.

We might want to expand on this in the future to normalize spacing between
parameters, and possibly remove the charset=utf-8; parameter. However, before
adding that complexity we should see how many clients actually fail to copy &
paste the expected content-type.

Bug: T128040
Change-Id: I70f9ffad5770c8a6d35f28f59c8b17dc4a396910
---
M templates/varnish/text-common.inc.vcl.erb
1 file changed, 6 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/42/281042/1

diff --git a/templates/varnish/text-common.inc.vcl.erb 
b/templates/varnish/text-common.inc.vcl.erb
index 044f58c..9836d9c 100644
--- a/templates/varnish/text-common.inc.vcl.erb
+++ b/templates/varnish/text-common.inc.vcl.erb
@@ -65,6 +65,12 @@
} else {
call normalize_mediawiki_path;
}
+
+   // Normalize Accept headers for the REST API: Ignore unless a profile is
+   // specified.
+   if (req.url ~ "^/api/rest_v1/" && req.http.Accept !~ 
"profile=\"https://www.mediawiki.org/wiki/Specs/;) {
+   unset req.http.Accept;
+   }
 }
 
 // fe+be common recv code

-- 
To view, visit https://gerrit.wikimedia.org/r/281042
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I70f9ffad5770c8a6d35f28f59c8b17dc4a396910
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] RESTBase: Increase log sampling rates - change (operations/puppet)

2016-04-01 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/280951

Change subject: RESTBase: Increase log sampling rates
..

RESTBase: Increase log sampling rates

We started sampling normal & slow request logs recently, which has proven very
useful for tracking down large request surges. With the current settings, we
are logging about 12-15 requests *per hour*, which is at the lower end of what
is useful.

This patch increases the sampling rate by a factor of five (and 10 for slow
requests). This should result in roughly one log message per minute.

Change-Id: I2a2a9982e684b146edfa7f429f7dfef4b5559a1b
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/51/280951/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 8b84fb4..39129eb 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -893,9 +893,9 @@
   level: <%= @logging_level %>
   sampled_levels:
 # Target is about 1 request logged per minute @1000 req/s
-trace/request: 0.1
+trace/request: 0.5
 # Not sure what the rate of these is, so start conservatively.
-trace/request/slow: 0.0001
+trace/request/slow: 0.001
   streams:
   # XXX: Use gelf-stream -> logstash
   - type: gelf

-- 
To view, visit https://gerrit.wikimedia.org/r/280951
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2a2a9982e684b146edfa7f429f7dfef4b5559a1b
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Start sampling regular & slow requests - change (operations/puppet)

2016-03-31 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/280711

Change subject: Start sampling regular & slow requests
..

Start sampling regular & slow requests

We need a better understanding of normal requests, without adding undue burden
on our logging infrastructure. To this end, this patch leverages
service-runner logger functionality to configure sampled logging of normal &
slow requests, aiming for a logging rate of no more than about one log message
per minute.

Change-Id: I972271b23b71cf5e954add6e7c92f37770ac6d77
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 5 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/11/280711/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index ecfb08c..8b84fb4 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -891,6 +891,11 @@
 logging:
   name: <%= @logging_name %>
   level: <%= @logging_level %>
+  sampled_levels:
+# Target is about 1 request logged per minute @1000 req/s
+trace/request: 0.1
+# Not sure what the rate of these is, so start conservatively.
+trace/request/slow: 0.0001
   streams:
   # XXX: Use gelf-stream -> logstash
   - type: gelf

-- 
To view, visit https://gerrit.wikimedia.org/r/280711
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I972271b23b71cf5e954add6e7c92f37770ac6d77
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Remove main node IPs from codfw restbase seeds - change (operations/puppet)

2016-03-30 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/280484

Change subject: Remove main node IPs from codfw restbase seeds
..

Remove main node IPs from codfw restbase seeds

restbase2004 is no longer running a Cassandra instance on the main node IP,
and the others will convert to multi-instance in the next weeks. The driver
auto-discovers nodes from seeds, so there is no need to list all cluster
nodes.

Change-Id: I800339f0493e6fb16a977b556a1ca750f9d977f5
---
M hieradata/role/codfw/restbase.yaml
M modules/varnishkafka
2 files changed, 0 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/84/280484/1

diff --git a/hieradata/role/codfw/restbase.yaml 
b/hieradata/role/codfw/restbase.yaml
index 63bad39..7ef2101 100644
--- a/hieradata/role/codfw/restbase.yaml
+++ b/hieradata/role/codfw/restbase.yaml
@@ -5,10 +5,6 @@
 - restbase2002-a.codfw.wmnet
 - restbase2002-b.codfw.wmnet
 - restbase2002-c.codfw.wmnet
-- restbase2003.codfw.wmnet
-- restbase2004.codfw.wmnet
-- restbase2005.codfw.wmnet
-- restbase2006.codfw.wmnet
 
 lvs::realserver::realserver_ips:
   - '10.2.1.17' # restbase.svc.codfw.wmnet
diff --git a/modules/varnishkafka b/modules/varnishkafka
index 982626a..a701155 16
--- a/modules/varnishkafka
+++ b/modules/varnishkafka
-Subproject commit 982626a59dd45dcf9240981c402daafeccc3b5ed
+Subproject commit a70115560f5c4410f9f3dcfeef2de6a9733807ad

-- 
To view, visit https://gerrit.wikimedia.org/r/280484
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I800339f0493e6fb16a977b556a1ca750f9d977f5
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Staging: Clean up restbase seeds - change (operations/puppet)

2016-03-30 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/280351

Change subject: Staging: Clean up restbase seeds
..

Staging: Clean up restbase seeds

A lot of IPs are in the list of cassandra seeds, but aren't actually running
cassandra instances yet / any more. This is creating a lot of log noise, as
clients are trying to connect to those non-existing instances.

This patch simply removes those inactive IPs.

Change-Id: I10ac2c23c74ff9eaa19f7e0fe67cb34a1c3251bc
---
M hieradata/regex.yaml
1 file changed, 0 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/51/280351/1

diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index 357c987..8ed2205 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -147,22 +147,13 @@
   - restbase-test2003-a.codfw.wmnet
   - restbase-test2003-b.codfw.wmnet
   restbase::seeds:
-  - cerium.eqiad.wmnet
   - cerium-a.eqiad.wmnet
-  - cerium-b.eqiad.wmnet
-  - praseodymium.eqiad.wmnet
   - praseodymium-a.eqiad.wmnet
-  - praseodymium-b.eqiad.wmnet
-  - xenon.eqiad.wmnet
   - xenon-a.eqiad.wmnet
-  - xenon-b.eqiad.wmnet
-  - restbase-test2001.codfw.wmnet
   - restbase-test2001-a.codfw.wmnet
   - restbase-test2001-b.codfw.wmnet
-  - restbase-test2002.codfw.wmnet
   - restbase-test2002-a.codfw.wmnet
   - restbase-test2002-b.codfw.wmnet
-  - restbase-test2003.codfw.wmnet
   - restbase-test2003-a.codfw.wmnet
   - restbase-test2003-b.codfw.wmnet
   restbase::cassandra_datacenters:

-- 
To view, visit https://gerrit.wikimedia.org/r/280351
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I10ac2c23c74ff9eaa19f7e0fe67cb34a1c3251bc
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 
Gerrit-Reviewer: Dzahn 
Gerrit-Reviewer: Eevans 
Gerrit-Reviewer: Filippo Giunchedi 
Gerrit-Reviewer: Mobrovac 
Gerrit-Reviewer: Ori.livneh 
Gerrit-Reviewer: Ppchelko 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Staging: Remove inactive IPs from Cassandra seeds - change (operations/puppet)

2016-03-30 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/280349

Change subject: Staging: Remove inactive IPs from Cassandra seeds
..

Staging: Remove inactive IPs from Cassandra seeds

A lot of IPs are in the list of cassandra seeds, but aren't actually running
cassandra instances yet / any more. This is creating a lot of log noise, as
clients are trying to connect to those non-existing instances.

This patch simply removes those inactive IPs. One potential issue could be the
coupling of firewalling with instance IPs. If outgoing connections from
restbase to remote cassandra nodes are using now-removed IPs (main node IP /
default route), then they might end up being blocked by the firewall.

Change-Id: I251ca6655fcf38387329187b63bab9b2f8de67ef
---
M hieradata/regex.yaml
1 file changed, 0 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/49/280349/1

diff --git a/hieradata/regex.yaml b/hieradata/regex.yaml
index 357c987..3ca1037 100644
--- a/hieradata/regex.yaml
+++ b/hieradata/regex.yaml
@@ -70,22 +70,13 @@
   __regex: !ruby/regexp /^(cerium|praseodymium|xenon)\.eqiad\.wmnet$/
   cluster: restbase_test
   cassandra::seeds:
-  - cerium.eqiad.wmnet
   - cerium-a.eqiad.wmnet
-  - cerium-b.eqiad.wmnet
-  - praseodymium.eqiad.wmnet
   - praseodymium-a.eqiad.wmnet
-  - praseodymium-b.eqiad.wmnet
-  - xenon.eqiad.wmnet
   - xenon-a.eqiad.wmnet
-  - xenon-b.eqiad.wmnet
-  - restbase-test2001.codfw.wmnet
   - restbase-test2001-a.codfw.wmnet
   - restbase-test2001-b.codfw.wmnet
-  - restbase-test2002.codfw.wmnet
   - restbase-test2002-a.codfw.wmnet
   - restbase-test2002-b.codfw.wmnet
-  - restbase-test2003.codfw.wmnet
   - restbase-test2003-a.codfw.wmnet
   - restbase-test2003-b.codfw.wmnet
   restbase::seeds:

-- 
To view, visit https://gerrit.wikimedia.org/r/280349
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I251ca6655fcf38387329187b63bab9b2f8de67ef
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Move eventbus topic config from operations/puppet to this repo - change (mediawiki/event-schemas)

2016-03-28 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Move eventbus topic config from operations/puppet to this repo
..


Move eventbus topic config from operations/puppet to this repo

Change-Id: Id50eaa554b9c0dfa29fcaa82fe633561e8593dc5
---
A config/eventbus-topics.yaml
1 file changed, 28 insertions(+), 0 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved
  jenkins-bot: Verified



diff --git a/config/eventbus-topics.yaml b/config/eventbus-topics.yaml
new file mode 100644
index 000..35b143f
--- /dev/null
+++ b/config/eventbus-topics.yaml
@@ -0,0 +1,28 @@
+
+#
+# Topic config file for eventbus produce service.
+# This file specifies which schemas are allowed to be produced
+# to topics.  Learn more at https://wikitech.wikimedia.org/wiki/EventBus
+#
+# Note: This topic config used to be in WMF's operations/puppet repo.
+# It may find a better home one day, but keeping it in this repository
+# allows it to be used by services outside of WMF production.
+#
+
+mediawiki.page_delete:
+  schema_name: page_delete
+
+mediawiki.page_move:
+  schema_name: page_move
+
+mediawiki.page_restore:
+  schema_name: page_restore
+
+mediawiki.revision_visibility_set:
+  schema_name: revision_visibility_set
+
+mediawiki.revision_create:
+  schema_name: revision_create
+
+resource_change:
+  schema_name: resource_change

-- 
To view, visit https://gerrit.wikimedia.org/r/280088
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Id50eaa554b9c0dfa29fcaa82fe633561e8593dc5
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/event-schemas
Gerrit-Branch: master
Gerrit-Owner: Ottomata 
Gerrit-Reviewer: GWicke 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Drop wmf prefix from resource_change topic - change (mediawiki/vagrant)

2016-03-25 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/279686

Change subject: Drop wmf prefix from resource_change topic
..

Drop wmf prefix from resource_change topic

This topic is not limited to WMF use, so it doesn't seem to make sense to
single it out as WMF-specific.

Change-Id: I23dba6b1b900694acd3de06279a6d3ad422be15d
---
M puppet/modules/role/files/eventbus/topics.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/86/279686/1

diff --git a/puppet/modules/role/files/eventbus/topics.yaml 
b/puppet/modules/role/files/eventbus/topics.yaml
index 090dcdf..e489419 100644
--- a/puppet/modules/role/files/eventbus/topics.yaml
+++ b/puppet/modules/role/files/eventbus/topics.yaml
@@ -20,5 +20,5 @@
 mediawiki.revision_create:
   schema_name: revision_create
 
-wmf.resource_change:
+resource_change:
   schema_name: resource_change

-- 
To view, visit https://gerrit.wikimedia.org/r/279686
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I23dba6b1b900694acd3de06279a6d3ad422be15d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Exclude null edits from edit metrics - change (mediawiki...WikimediaEvents)

2016-03-25 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/279681

Change subject: Exclude null edits from edit metrics
..

Exclude null edits from edit metrics

Edit metrics are generally interested in actual edits with content changes,
rather than null edits without any content change.

To this end, the WikimediaEventsHooks extension had set up a quick return for
the case that the $revision parameter is null. This is inspired by the
documentation for PageContentSaveComplete [1], which mentions that the
$revision parameter *can* be null if the content was not changed. However, on
the MediaWiki side, the $revision parameter actually ends up being defined
even if the content has not changed.

This patch improves null edit detection by additionally checking for the
revision property in the status object, which is null for null edits.

[1]: https://www.mediawiki.org/wiki/Manual:Hooks/PageContentSaveComplete

Bug: T128838
Change-Id: Ia855b653534a19c35f600988f59e204457cad3c4
---
M WikimediaEventsHooks.php
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/WikimediaEvents 
refs/changes/81/279681/1

diff --git a/WikimediaEventsHooks.php b/WikimediaEventsHooks.php
index 7f70a97..e74e23f 100644
--- a/WikimediaEventsHooks.php
+++ b/WikimediaEventsHooks.php
@@ -54,7 +54,8 @@
public static function onPageContentSaveComplete( $article, $user, 
$content, $summary,
$isMinor, $isWatch, $section, $flags, $revision, $status, 
$baseRevId ) {
 
-   if ( !$revision ) {
+   if ( !$revision || is_null( $status->getValue()['revision'] ) ) 
{
+   // Null edit. Do not count.
return;
}
 

-- 
To view, visit https://gerrit.wikimedia.org/r/279681
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia855b653534a19c35f600988f59e204457cad3c4
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/WikimediaEvents
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] WIP: Rewrite /static/ also for PURGE requests - change (operations/puppet)

2016-03-24 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/279564

Change subject: WIP: Rewrite /static/ also for PURGE requests
..

WIP: Rewrite /static/ also for PURGE requests

Bug: T130904
Change-Id: I6ac59be92ad171332c81987e9153be8266ba629e
---
M templates/varnish/text-backend.inc.vcl.erb
M templates/varnish/text-frontend.inc.vcl.erb
2 files changed, 18 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/64/279564/1

diff --git a/templates/varnish/text-backend.inc.vcl.erb 
b/templates/varnish/text-backend.inc.vcl.erb
index d07c220..ed987d4 100644
--- a/templates/varnish/text-backend.inc.vcl.erb
+++ b/templates/varnish/text-backend.inc.vcl.erb
@@ -5,6 +5,15 @@
 sub cluster_be_recv_pre_purge {
if (req.request == "PURGE") {
call text_normalize_path;
+
+   # normalize all /static to the same hostname for caching
+   if (req.url ~ "^/static/") { set req.http.host = "<%= 
@vcl_config.fetch("static_host") %>"; }
+
+   # normalize all /w/static.php to the same wiki host for caching
+   # ignore urls without hash query as those are affected by 
multiversion
+   if (req.url ~ 
"^/w/(skins|resources|extensions)/.+\?[a-fA-F0-9]+$" ) {
+   set req.http.host = "<%= 
@vcl_config.fetch("static_host") %>";
+   }
}
 }
 
diff --git a/templates/varnish/text-frontend.inc.vcl.erb 
b/templates/varnish/text-frontend.inc.vcl.erb
index 098fa53..c601533 100644
--- a/templates/varnish/text-frontend.inc.vcl.erb
+++ b/templates/varnish/text-frontend.inc.vcl.erb
@@ -113,6 +113,15 @@
 
// Normalize paths before purging
call text_normalize_path;
+
+   # normalize all /static to the same hostname for caching
+   if (req.url ~ "^/static/") { set req.http.host = "<%= 
@vcl_config.fetch("static_host") %>"; }
+
+   # normalize all /w/static.php to the same wiki host for caching
+   # ignore urls without hash query as those are affected by multiversion
+   if (req.url ~ "^/w/(skins|resources|extensions)/.+\?[a-fA-F0-9]+$" ) {
+   set req.http.host = "<%= @vcl_config.fetch("static_host") %>";
+   }
 }
 
 sub cluster_fe_recv {
@@ -125,15 +134,6 @@
}
 
call mobile_redirect;
-
-   # normalize all /static to the same hostname for caching
-   if (req.url ~ "^/static/") { set req.http.host = "<%= 
@vcl_config.fetch("static_host") %>"; }
-
-   # normalize all /w/static.php to the same wiki host for caching
-   # ignore urls without hash query as those are affected by multiversion
-   if (req.url ~ "^/w/(skins|resources|extensions)/.+\?[a-fA-F0-9]+$" ) {
-   set req.http.host = "<%= @vcl_config.fetch("static_host") %>";
-   }
 
// Users that just logged out, should not get a 304 for their
// (locally cached) logged in pages.

-- 
To view, visit https://gerrit.wikimedia.org/r/279564
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6ac59be92ad171332c81987e9153be8266ba629e
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Increase purged entry point s-maxage from 12 to 48 hours - change (operations/puppet)

2016-03-13 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/277112

Change subject: Increase purged entry point s-maxage from 12 to 48 hours
..

Increase purged entry point s-maxage from 12 to 48 hours

We have not encountered any issues with a TTL of 12 hours, and have deployed
code which disables caching for all requests with non-canonical titles.

It should thus be safe to take the next step towards longer TTLs. 48 hours
represents a conservative increment on the way to 1-2 weeks.

Change-Id: Ifca932d1efe8ea829a84dc5e9039c89ee69f74d3
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/12/277112/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 5a4ebc7..8e162fa 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -61,7 +61,7 @@
 related:
   cache_control: s-maxage=86400, max-age=3600
 # Cache headers for purged end points.
-purged_cache_control: s-maxage=43200, max-age=0, must-revalidate
+purged_cache_control: s-maxage=172800, max-age=0, must-revalidate
 
 # A different project template, sharing configuration options.
 wikimedia.org: 

-- 
To view, visit https://gerrit.wikimedia.org/r/277112
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifca932d1efe8ea829a84dc5e9039c89ee69f74d3
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] RESTBase caching: Force clients to revalidate purged end points - change (operations/puppet)

2016-03-12 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/277056

Change subject: RESTBase caching: Force clients to revalidate purged end points
..

RESTBase caching: Force clients to revalidate purged end points

Most of our purged end points should be re-validated by clients on each view,
so that they make sure to have the latest version before reusing content. This
forces clients to re-validate by setting max-age to 0, and adding
must-revalidate.

This patch also reduces max-age for the related end point to one hour, while
leaving s-maxage at a day. This makes sure that clients get the latest copy
from the CDN every now & then, and reduces the time to clear caches in
emergencies.

Change-Id: I16c494b2178e82364bda763662d604c49641efa1
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/56/277056/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index a060acb..5a4ebc7 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -59,9 +59,9 @@
 host: <%= @purge_host %>
 port: <%= @purge_port %>
 related:
-  cache_control: s-maxage=86400, max-age=86400
+  cache_control: s-maxage=86400, max-age=3600
 # Cache headers for purged end points.
-purged_cache_control: s-maxage=43200, max-age=43200
+purged_cache_control: s-maxage=43200, max-age=0, must-revalidate
 
 # A different project template, sharing configuration options.
 wikimedia.org: 

-- 
To view, visit https://gerrit.wikimedia.org/r/277056
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I16c494b2178e82364bda763662d604c49641efa1
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Performance: Only pull in the core-js shim for node < v4 - change (mediawiki...parsoid)

2016-03-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/276697

Change subject: Performance: Only pull in the core-js shim for node < v4
..

Performance: Only pull in the core-js shim for node < v4

Several methods from core-js showed up high in profiles, especially the
wrapped JSON.stringify, and fix-re-wks. Since none of the advanced core-js
functionality seems to be actually needed when using node >= v4, this patch
simply avoids installing the global core-js shim for node >= 4.

In limited benchmarking over wifi / cable, this speeds up parses of Obama from
around 17.5s to 12.6s. Memory usage is also reduced significantly.

Bug: T129598
Change-Id: I9a6ef0f97c6150ea84abce3e94115c016a43db48
---
M core-upgrade.js
1 file changed, 3 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/97/276697/1

diff --git a/core-upgrade.js b/core-upgrade.js
index 413ac4b..8ac87a7 100644
--- a/core-upgrade.js
+++ b/core-upgrade.js
@@ -1,2 +1,4 @@
 'use strict';
-require('core-js/shim');
+if (process.version < 'v4.') {
+   require('core-js/shim');
+}

-- 
To view, visit https://gerrit.wikimedia.org/r/276697
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9a6ef0f97c6150ea84abce3e94115c016a43db48
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Use underscores in titles sent to the RB / Parsoid API - change (mediawiki...bundler)

2016-03-07 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/275746

Change subject: Use underscores in titles sent to the RB / Parsoid API
..

Use underscores in titles sent to the RB / Parsoid API

We are starting to normalize titles to consistently use underscores, and will
soon start redirecting any non-canonical requests to their canonical
equivalent.

This patch makes OCG request titles with underscores instead of spaces.

Change-Id: I7dcd46f32d6534bfb8031f10c44d22dacc57a905
---
M lib/parsoid.js
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Collection/OfflineContentGenerator/bundler
 refs/changes/46/275746/1

diff --git a/lib/parsoid.js b/lib/parsoid.js
index 659e69c..d9bb13e 100644
--- a/lib/parsoid.js
+++ b/lib/parsoid.js
@@ -143,7 +143,8 @@
} else {
apiURL = url.resolve(apiURL, './page/html/');
}
-   apiURL += encodeURIComponent(title); /* Encode slashes. */
+// Use underscores, escape delimiters incl. slashes.
+   apiURL += encodeURIComponent(title.replace(/ /g, '_'));
if (revid) {
apiURL += '/' + revid;
}

-- 
To view, visit https://gerrit.wikimedia.org/r/275746
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7dcd46f32d6534bfb8031f10c44d22dacc57a905
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/Collection/OfflineContentGenerator/bundler
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] RESTBase: Increase purged cache TTL to 12 hours - change (operations/puppet)

2016-03-03 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/274799

Change subject: RESTBase: Increase purged cache TTL to 12 hours
..

RESTBase: Increase purged cache TTL to 12 hours

Tests after the deploy of 274769 indicate that purging is now working
well for titles with special characters. This means that we can start to
gradually increase the cache TTL from the very conservative 1h towards more
reasonable values for cached entry points.

To be on the safe side, this patch bumps the TTLs to 12 hours. This should be
long enough to surface purging issues (if there are any left), but is also
still short enough to let us react in a reasonable time window.

Change-Id: Ie0382b70bb0d43947cf79a91e157968a6f495908
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/99/274799/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 09558a9..b1d94e8 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -59,7 +59,7 @@
 host: <%= @purge_host %>
 port: <%= @purge_port %>
 # Cache headers for purged end points.
-purged_cache_control: s-maxage=3600, max-age=3600
+purged_cache_control: s-maxage=43200, max-age=43200
 
 # A different project template, sharing configuration options.
 wikimedia.org: 

-- 
To view, visit https://gerrit.wikimedia.org/r/274799
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie0382b70bb0d43947cf79a91e157968a6f495908
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Start to ramp up cache TTL for purged RESTBase end points - change (operations/puppet)

2016-03-03 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/274769

Change subject: Start to ramp up cache TTL for purged RESTBase end points
..

Start to ramp up cache TTL for purged RESTBase end points

With path normalization now applied to both GETs and PURGEs, we should have
working purging for titles with special characters. This means that we can
start to ramp up TTLs again.

This patch is only bumping the TTL to one hour for now. Once we are sure that
purging is working as expected, we can follow up with further TTL increases,
eventually matching the number of weeks generally used for purged content.

Change-Id: Ia6dbf94fb5b0e3bdb1cc1735fedeca65eff21a50
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/69/274769/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 719fad6..09558a9 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -59,7 +59,7 @@
 host: <%= @purge_host %>
 port: <%= @purge_port %>
 # Cache headers for purged end points.
-purged_cache_control: s-maxage=1, max-age=1
+purged_cache_control: s-maxage=3600, max-age=3600
 
 # A different project template, sharing configuration options.
 wikimedia.org: 

-- 
To view, visit https://gerrit.wikimedia.org/r/274769
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia6dbf94fb5b0e3bdb1cc1735fedeca65eff21a50
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] WIP: Add a cluster_be_recv_pre_purge handler & normalize paths - change (operations/puppet)

2016-03-02 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/274458

Change subject: WIP: Add a cluster_be_recv_pre_purge handler & normalize paths
..

WIP: Add a cluster_be_recv_pre_purge handler & normalize paths

To ensure reliable purging of titles with special characters, we would like to
apply the same path normalizations to PURGE and other requests. The previous
attempt to do so (274282) only applied to frontends, so was not a complete
solution.

This patch introduces an equivalent cluster_be_recv_pre_purge handler, and
calls / defines it for all Varnish clusters. For the text cluster, this
handler is then used to apply the same path normalizations as performed in the
frontend to PURGE requests.

Bug: T127387
Change-Id: I9a882013adb620ae5f5e36eb0ad4ef1a9ba1b0a0
---
M modules/varnish/templates/vcl/wikimedia-backend.vcl.erb
M templates/varnish/maps-backend.inc.vcl.erb
M templates/varnish/misc-backend.inc.vcl.erb
M templates/varnish/text-backend.inc.vcl.erb
M templates/varnish/text-common.inc.vcl.erb
M templates/varnish/text-frontend.inc.vcl.erb
M templates/varnish/upload-backend.inc.vcl.erb
7 files changed, 27 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/58/274458/1

diff --git a/modules/varnish/templates/vcl/wikimedia-backend.vcl.erb 
b/modules/varnish/templates/vcl/wikimedia-backend.vcl.erb
index 6329f5e..d429804 100644
--- a/modules/varnish/templates/vcl/wikimedia-backend.vcl.erb
+++ b/modules/varnish/templates/vcl/wikimedia-backend.vcl.erb
@@ -11,6 +11,7 @@
}
 
call wm_common_recv;
+   call cluster_be_recv_pre_purge;
call wm_common_recv_purge;
call cluster_be_recv;
 
diff --git a/templates/varnish/maps-backend.inc.vcl.erb 
b/templates/varnish/maps-backend.inc.vcl.erb
index b9b4df6..8b66812 100644
--- a/templates/varnish/maps-backend.inc.vcl.erb
+++ b/templates/varnish/maps-backend.inc.vcl.erb
@@ -1,4 +1,5 @@
 // Varnish VCL include file for maps backends
+sub cluster_be_recv_pre_purge { }
 sub cluster_be_recv { }
 sub cluster_be_hash { }
 sub cluster_be_hit { }
diff --git a/templates/varnish/misc-backend.inc.vcl.erb 
b/templates/varnish/misc-backend.inc.vcl.erb
index 24f76c0..ed9e315 100644
--- a/templates/varnish/misc-backend.inc.vcl.erb
+++ b/templates/varnish/misc-backend.inc.vcl.erb
@@ -1,5 +1,7 @@
 include "misc-common.inc.vcl";
 
+sub cluster_be_recv_pre_purge { }
+
 sub cluster_be_recv {
 <% if @site_tier == "one" -%>
 if (req.http.Host == "git.wikimedia.org") {
diff --git a/templates/varnish/text-backend.inc.vcl.erb 
b/templates/varnish/text-backend.inc.vcl.erb
index fccd248..629c103 100644
--- a/templates/varnish/text-backend.inc.vcl.erb
+++ b/templates/varnish/text-backend.inc.vcl.erb
@@ -1,6 +1,13 @@
 // Varnish VCL include file for text backends
 
 include "text-common.inc.vcl";
+include "normalize_path.inc.vcl";
+
+sub cluster_be_recv_pre_purge { 
+   if (req.request == "PURGE") {
+   call text_normalize_path;
+   }
+}
 
 sub cluster_be_recv {
 <% if @site_tier == "one" -%>
diff --git a/templates/varnish/text-common.inc.vcl.erb 
b/templates/varnish/text-common.inc.vcl.erb
index 53e3dcf..c825adf 100644
--- a/templates/varnish/text-common.inc.vcl.erb
+++ b/templates/varnish/text-common.inc.vcl.erb
@@ -1,5 +1,7 @@
 // Common functions for the Text/Mobile Varnish clusters
 
+include "normalize_path.inc.vcl";
+
 sub text_common_misspass_restore_cookie {
// Restore the original Cookie header for upstream
// Assumes client header X-Orig-Cookie has been filtered!
@@ -56,6 +58,15 @@
}
 }
 
+sub text_normalize_path {
+   // Don't decode percent-encoded slashes in paths for REST APIs
+   if (req.url ~ "^/api/rest_v1/" || req.http.host ~ 
"cxserver|citoid|restbase|^rest\.") {
+   call normalize_rest_path;
+   } else {
+   call normalize_mediawiki_path;
+   }
+}
+
 // fe+be common recv code
 sub text_common_recv {
if (req.request != "GET" && req.request != "HEAD") {
diff --git a/templates/varnish/text-frontend.inc.vcl.erb 
b/templates/varnish/text-frontend.inc.vcl.erb
index 7d7b7fc..b295ae3 100644
--- a/templates/varnish/text-frontend.inc.vcl.erb
+++ b/templates/varnish/text-frontend.inc.vcl.erb
@@ -3,7 +3,6 @@
 include "text-common.inc.vcl";
 include "zero.inc.vcl";
 include "geoip.inc.vcl";
-include "normalize_path.inc.vcl";
 
 // Note that analytics.inc.vcl will set an X-Analytics value of proxy=IORG
 // without inspecting whether there's an existing proxy= key-
@@ -111,6 +110,9 @@
}
}
}
+
+   // Normalize paths before purging
+   call text_normalize_path;
 }
 
 sub cluster_fe_recv {
@@ -120,13 +122,6 @@
error 204;
}
return (lookup);
-   }
-
-   // Don't decode percent-encoded slashes in paths for

[MediaWiki-commits] [Gerrit] Temporarily reduce cache timeout until purging works - change (operations/puppet)

2016-03-02 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/274456

Change subject: Temporarily reduce cache timeout until purging works
..

Temporarily reduce cache timeout until purging works

Until T127387 is resolved, purging does not work reliably for titles that
contain special characters such as [:;()@].

Today's deploy included a patch that enables caching (and purging) for
additional end points, some of which are used by VisualEditor to retrieve the
latest HTML for a page. While VE has code to detect stale responses, at the
minimum this slows down the overall response. Other clients do not have such
logic, and will see outdated content.

To avoid such issues, this patch temporarily lowers the cache timeout to one
second. A patch to fix purging is likely to be ready later today or tomorrow,
and temporarily reducing caching should not cause significant performance
issues at this point.

Change-Id: I65c71f5b52f15ec2c3a61bca40446efb45200921
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/56/274456/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 09558a9..719fad6 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -59,7 +59,7 @@
 host: <%= @purge_host %>
 port: <%= @purge_port %>
 # Cache headers for purged end points.
-purged_cache_control: s-maxage=3600, max-age=3600
+purged_cache_control: s-maxage=1, max-age=1
 
 # A different project template, sharing configuration options.
 wikimedia.org: 

-- 
To view, visit https://gerrit.wikimedia.org/r/274456
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I65c71f5b52f15ec2c3a61bca40446efb45200921
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add purged_cache_control config variable - change (operations/puppet)

2016-02-29 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/273974

Change subject: Add purged_cache_control config variable
..

Add purged_cache_control config variable

https://github.com/wikimedia/restbase/pull/520 will use this variable to
centrally configure cache headers for all purged entry points. Until that is
deployed, this config variable is not referenced by any other code, so can be
safely deployed.

Change-Id: Ifea659001f45e0d6cdb8a5502ed0efefce933b38
---
M modules/restbase/templates/config.labs.yaml.erb
M modules/restbase/templates/config.yaml.erb
2 files changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/74/273974/1

diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index 19ec7b8..9cc18b7 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -35,6 +35,8 @@
   purge:
 host: <%= @purge_host %>
 port: <%= @purge_port %>
+# Cache headers for purged end points.
+purged_cache_control: s-maxage=3600, max-age=3600
 
 # A different project template, sharing configuration options.
 wikimedia.org: 
diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 2ae7d4d..09558a9 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -58,6 +58,8 @@
   purge:
 host: <%= @purge_host %>
 port: <%= @purge_port %>
+# Cache headers for purged end points.
+purged_cache_control: s-maxage=3600, max-age=3600
 
 # A different project template, sharing configuration options.
 wikimedia.org: 

-- 
To view, visit https://gerrit.wikimedia.org/r/273974
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifea659001f45e0d6cdb8a5502ed0efefce933b38
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] REST path escaping - change (operations/puppet)

2016-02-24 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/273146

Change subject: REST path escaping
..

REST path escaping

REST APIs like their slashes preserved, while MediaWiki likes them decoded.
This patch adds a REST API normalization variant that decodes everything but
the path delimiters [/?#] as well as % itself. This includes the `unreserved`
set and `sub-delim` classes from RFC3986, as well as those characters from
`gen-delim` that are only used in the host / auth portion of a URL ([:@]).

As discussed on the task, the MediaWiki variant should probably include at
least `unreserved` and much of `gen-delim` as well. This patch does not touch
this just yet, leaving it to a follow-up once consensus on those changes is
reached.

Change-Id: I23d5baf099e4a04aa451efde7cd7d952c202e6cd
---
M templates/varnish/text-frontend.inc.vcl.erb
1 file changed, 78 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/46/273146/1

diff --git a/templates/varnish/text-frontend.inc.vcl.erb 
b/templates/varnish/text-frontend.inc.vcl.erb
index 93c197e..6d63d94 100644
--- a/templates/varnish/text-frontend.inc.vcl.erb
+++ b/templates/varnish/text-frontend.inc.vcl.erb
@@ -50,7 +50,7 @@
 C{
#include 
 }C
-sub normalize_path {
+sub normalize_mediawiki_path {
/* Rewrite the path part of the URL, replacing unnecessarily escaped
 * punctuation with the actual characters. The character list is from
 * MediaWiki's wfUrlencode(), so the URLs produced here will be the 
same as
@@ -107,6 +107,79 @@
/* Reached the query part. Just copy 
the rest of the URL
 * to the destination.
 */
+   memcpy(destBuffer + outPos, url + i, 
sizeof(char) * (urlLength - i));
+   outPos += urlLength - i;
+   i = urlLength;
+   } else {
+   destBuffer[outPos++] = url[i];
+   }
+   }
+   destBuffer[outPos] = '\0';
+
+   /* Set req.url. This will copy our stack buffer into 
the workspace.
+* VRT_l_req_url() is varadic, and concatenates its 
arguments. The
+* vrt_magic_string_end marks the end of the list.
+*/
+   if (dirty) {
+   VRT_l_req_url(sp, destBuffer, 
vrt_magic_string_end);
+   }
+   }
+   #undef NP_IS_HEX
+   #undef NP_HEX_DIGIT
+   #undef NP_HEXCHAR
+   }C
+}
+
+sub normalize_rest_path {
+   /* Rewrite the path part of the URL, replacing unnecessarily escaped
+* punctuation with the actual characters. The character list is based 
on
+* the discussion in T127387, aiming for typical REST API use cases,
+* including RESTBase. */
+   C{
+   /* DIY hexadecimal conversion, since it is simple enough for a 
fixed
+* width, and all the relevant standard C library functions 
promise to
+* malfunction if the locale is set to anything other than "C"
+*/
+   #define NP_HEX_DIGIT(c) ( \
+   (c) >= '0' && (c) <= '9' ? (c) - '0' : ( \
+   (c) >= 'A' && (c) <= 'F' ? (c) - 'A' + 0x0a : ( 
\
+   (c) >= 'a' && (c) <= 'f' ? (c) - 'a' + 
0x0a : -1 ) ) )
+   #define NP_IS_HEX(c) (NP_HEX_DIGIT(c) != -1)
+   #define NP_HEXCHAR(c1, c2) (char)( (NP_HEX_DIGIT(c1) << 4) | 
NP_HEX_DIGIT(c2) )
+   const char * url = VRT_r_req_url(sp);
+   size_t i, outPos;
+   const size_t urlLength = strlen(url);
+// index for the last position %XX can start at:
+   const size_t lastConvertIdx = urlLength > 2 ? urlLength - 3 : 0;
+   char c;
+   int dirty = 0;
+
+   /* Allocate destination memory from the stack using the C99
+* variable-length automatic feature. We know the length in 
advance
+* because this function can only shorten the input string.
+*/
+   char destBuffer[urlLength + 1];
+   if (url) {
+   for (i = 0, outPos = 0; i < urlLength; i++) {
+   if (i <= lastConvertIdx && url[i] == '%' && 
NP_IS_HEX(url[i+1]) && NP_IS_HEX(url[i+2])) {
+   c = NP_HEXCHAR(url[i+1], url[i+2]);
+   /* Don't decode the subset of 
gen-delims from RFC3986
+

[MediaWiki-commits] [Gerrit] Update restbase to 3ae3cca - change (mediawiki...deploy)

2016-02-16 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/271194

Change subject: Update restbase to 3ae3cca
..

Update restbase to 3ae3cca

List of changes:
xxx Update node module dependencies

Change-Id: I5fcbdef0115708c1207b4d2c37c18558b9ad67f0
---
M node_modules/content-type/package.json
M node_modules/hyperswitch/lib/server.js
M node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
M node_modules/hyperswitch/node_modules/swagger-ui/package.json
M node_modules/hyperswitch/package.json
M node_modules/json-stable-stringify/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/json-stable-stringify/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/routeswitch/node_modules/async/package.json
A node_modules/service-runner/node_modules/hot-shots/CHANGES.md~
9 files changed, 64 insertions(+), 24 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/94/271194/1

diff --git a/node_modules/content-type/package.json 
b/node_modules/content-type/package.json
index c8860ca..716bfb8 100644
--- a/node_modules/content-type/package.json
+++ b/node_modules/content-type/package.json
@@ -44,7 +44,7 @@
   },
   "homepage": "https://github.com/jshttp/content-type#readme;,
   "_id": "content-type@1.0.1",
-  "_shasum": "bf8e3dd9b3ab4be91b02c7cc7e7b9a0efb0da680",
+  "_shasum": "7a614bb197ff960bf11d9fa84b8214edcecfaf0c",
   "_from": "git+https://github.com/wikimedia/content-type.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/content-type.git#47b2632d0a2ee79a7d67268e2f6621becd95d05b;
 }
diff --git a/node_modules/hyperswitch/lib/server.js 
b/node_modules/hyperswitch/lib/server.js
index d1cbfa8..7333f97 100644
--- a/node_modules/hyperswitch/lib/server.js
+++ b/node_modules/hyperswitch/lib/server.js
@@ -327,6 +327,8 @@
 } else {
 reqOpts.metrics = opts.child_metrics.internal;
 }
+} else {
+reqOpts.metrics = opts.child_metrics.external;
 }
 
 // Create a new, clean request object
@@ -410,8 +412,9 @@
 log: options.logger && options.logger.log.bind(options.logger) || 
function() {},
 metrics: options.metrics,
 child_metrics: {
-internal: options.metrics && options.metrics.makeChild('private'),
-internal_update: options.metrics && 
options.metrics.makeChild('internal_update')
+external: options.metrics && options.metrics.makeChild('external'),
+internal: options.metrics && options.metrics.makeChild('internal'),
+internal_update: options.metrics && 
options.metrics.makeChild('internal_update'),
 }
 };
 
@@ -452,4 +455,4 @@
 main();
 } else {
 module.exports = main;
-}
\ No newline at end of file
+}
diff --git 
a/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json 
b/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
index da87296..1f4ef5a 100644
--- a/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
+++ b/node_modules/hyperswitch/node_modules/json-stable-stringify/package.json
@@ -50,7 +50,7 @@
 "url": "https://github.com/substack/json-stable-stringify/issues;
   },
   "_id": "json-stable-stringify@1.0.0",
-  "_shasum": "60d7db079b0cdf277ef09c8845bca99400ab08bb",
+  "_shasum": "b0fb91bda63ca2b5c71cf3b4af7d95301708357f",
   "_from": "git+https://github.com/wikimedia/json-stable-stringify.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/json-stable-stringify.git#c05178e4dfc3834017f017732934c1c52ba3dd8f;
 }
diff --git a/node_modules/hyperswitch/node_modules/swagger-ui/package.json 
b/node_modules/hyperswitch/node_modules/swagger-ui/package.json
index 174a99e..9d21c94 100644
--- a/node_modules/hyperswitch/node_modules/swagger-ui/package.json
+++ b/node_modules/hyperswitch/node_modules/swagger-ui/package.json
@@ -60,7 +60,7 @@
 "url": "https://github.com/swagger-api/swagger-ui/issues;
   },
   "_id": "swagger-ui@2.1.4",
-  "_shasum": "b0bad636fd37bea514a4078fb8fb47a5b913e7e3",
+  "_shasum": "789fb198f32d0a5ec3d7f9ce84a69c6665ae4446",
   "_from": "git+https://github.com/wikimedia/swagger-ui.git#master;,
   "_resolved": 
"git+https://github.com/wikimedia/swagger-ui.git#7bef30a93a3b81c55cb95d3ce9e0f9f53d5d8262;
 }
diff --git a/node_modules/hyperswitch/package.json 
b/node_modules/hyperswitch/package.json
index 82f6e59..0781365 100644
--- a/node_modules/hyperswitch/package.json
+++ b/node_modules/hyperswitch/package.json
@@ -1,6 +1,6 @@
 {
   "name": "hyperswitch",
-  "version": "0.1.4",
+  "version": "0.1.5",
   "description": "REST API creation framework",
   "main": "index.js",
   "scripts": {
@@ -35,8 +35,8 @@
 "cassandra-uuid": "^0.0.2",
 "preq": "^0.4.8",
 "swagger-router": "^0.4.0",
-"swagger-ui":

[MediaWiki-commits] [Gerrit] WIP / untested: Don't decode percent encoding for rest.wikim... - change (operations/puppet)

2016-01-29 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/267381

Change subject: WIP / untested: Don't decode percent encoding for 
rest.wikimedia.org
..

WIP / untested: Don't decode percent encoding for rest.wikimedia.org

Bug: T125176
Change-Id: I80880d8a644dbc98e783cb8aed1ed711e01b09bf
---
M templates/varnish/text-frontend.inc.vcl.erb
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/81/267381/1

diff --git a/templates/varnish/text-frontend.inc.vcl.erb 
b/templates/varnish/text-frontend.inc.vcl.erb
index bd21f29..bd4ed09 100644
--- a/templates/varnish/text-frontend.inc.vcl.erb
+++ b/templates/varnish/text-frontend.inc.vcl.erb
@@ -225,7 +225,7 @@
}
 
// Don't decode percent-encoded slashes in paths for REST APIs
-   if (req.url !~ "^/api/rest_v1/") {
+   if (req.url !~ "^/api/rest_v1/" && req.http.host !~ 
"rest.wikimedia.org") {
call normalize_path;
}
 

-- 
To view, visit https://gerrit.wikimedia.org/r/267381
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I80880d8a644dbc98e783cb8aed1ed711e01b09bf
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Do not hardcode the IP of the internal API LVS entry point - change (mediawiki...deploy)

2016-01-28 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/267050

Change subject: Do not hardcode the IP of the internal API LVS entry point
..

Do not hardcode the IP of the internal API LVS entry point

Change-Id: I133bc9942c0416a42433d1f4014703db5be65722
---
M conf/wmf/localsettings.js
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid/deploy 
refs/changes/50/267050/1

diff --git a/conf/wmf/localsettings.js b/conf/wmf/localsettings.js
index 1c51b0a..693ac62 100644
--- a/conf/wmf/localsettings.js
+++ b/conf/wmf/localsettings.js
@@ -10,7 +10,7 @@
 
// Use the API backends directly without hitting the text varnishes.
// API requests are not cacheable anyway.
-   parsoidConfig.defaultAPIProxyURI = 'http://10.2.2.22';
+   parsoidConfig.defaultAPIProxyURI = 'http://api.svc.eqiad.wmnet';
 
// Allow cors as we are only running on non-sensitive domains
parsoidConfig.allowCORS = '*';

-- 
To view, visit https://gerrit.wikimedia.org/r/267050
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I133bc9942c0416a42433d1f4014703db5be65722
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Remove the Parsoid cache - change (mediawiki...deploy)

2016-01-28 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/267049

Change subject: Remove the Parsoid cache
..

Remove the Parsoid cache

The Parsoid cache is decommissioned, and Parsoid should not try to use stale
HTML content from the cache any more in any case.

This patch removes the reference to the cache completely.

Change-Id: I0dbdfb5c6c720fef27e472e64faf1c566af69706
---
M conf/wmf/localsettings.js
M src
2 files changed, 0 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid/deploy 
refs/changes/49/267049/1

diff --git a/conf/wmf/localsettings.js b/conf/wmf/localsettings.js
index eb5f6de..1c51b0a 100644
--- a/conf/wmf/localsettings.js
+++ b/conf/wmf/localsettings.js
@@ -7,8 +7,6 @@
parsoidConfig.useBatchAPI = true;
 
parsoidConfig.useSelser = true;
-   // parsoid cache url
-   parsoidConfig.parsoidCacheURI = 'http://10.2.2.29/';
 
// Use the API backends directly without hitting the text varnishes.
// API requests are not cacheable anyway.
diff --git a/src b/src
index f1ddfb8..500f52a 16
--- a/src
+++ b/src
-Subproject commit f1ddfb884e32715c8b16d5149ee9b5119fc7de74
+Subproject commit 500f52a6629dc212906d128e1d2fca169186aeaa

-- 
To view, visit https://gerrit.wikimedia.org/r/267049
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0dbdfb5c6c720fef27e472e64faf1c566af69706
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Varnish: Don't disable caching for authenticated REST API re... - change (operations/puppet)

2015-12-30 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/261662

Change subject: Varnish: Don't disable caching for authenticated REST API 
requests
..

Varnish: Don't disable caching for authenticated REST API requests

Bug: T122673
Change-Id: Ibfd16c0bf181dad1678a3fe11ce9f009b0c267d5
---
M templates/varnish/text-common.inc.vcl.erb
1 file changed, 5 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/62/261662/1

diff --git a/templates/varnish/text-common.inc.vcl.erb 
b/templates/varnish/text-common.inc.vcl.erb
index 518d202..cc92bdf 100644
--- a/templates/varnish/text-common.inc.vcl.erb
+++ b/templates/varnish/text-common.inc.vcl.erb
@@ -20,11 +20,13 @@
 sub evaluate_cookie {
// A session or token cookie will prevent caching if varied on,
// move the Cookie header out of the way otherwise.
-   // Note static and load.php exceptions to the user/token
-   // exceptions, as these should be cacheable even when logged in.
+   // Note static, load.php and REST API exceptions, as these should be
+   // cacheable even when logged in.
+   // FIXME: This ignores Vary completely.
if (req.http.Cookie ~ "([sS]ession|Token)="
&& req.url !~ "^/static/"
-   && req.url !~ "^/w/load\.php") {
+   && req.url !~ "^/w/load\.php"
+   && req.url !~ "^/api/rest_v1/") {
return (pass);
} else {
call stash_cookie;

-- 
To view, visit https://gerrit.wikimedia.org/r/261662
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibfd16c0bf181dad1678a3fe11ce9f009b0c267d5
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Release v0.3.0 - change (mediawiki...mathoid)

2015-12-21 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/260532

Change subject: Release v0.3.0
..

Release v0.3.0

Change-Id: I197d5f05f4c3112291bf761c2dc674c974c9490d
---
M package.json
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/mathoid 
refs/changes/32/260532/1

diff --git a/package.json b/package.json
index c2aa217..0d5dcbe 100644
--- a/package.json
+++ b/package.json
@@ -1,6 +1,6 @@
 {
   "name": "mathoid",
-  "version": "0.2.9",
+  "version": "0.3.0",
   "description": "Render TeX to SVG and MathML using MathJax. Based on 
svgtex.",
   "main": "./app.js",
   "scripts": {

-- 
To view, visit https://gerrit.wikimedia.org/r/260532
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I197d5f05f4c3112291bf761c2dc674c974c9490d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/mathoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Reduce the number of restbase runners to limit parallelism - change (operations/puppet)

2015-12-15 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/259309

Change subject: Reduce the number of restbase runners to limit parallelism
..

Reduce the number of restbase runners to limit parallelism

https://gerrit.wikimedia.org/r/#/c/259207/ increased the number of job runner
nodes by about 54%, which implicitly increases the parallelism & thus the
potential peak load on Parsoid and related infrastructure. This patch reduces
the number of job runner workers per node slightly to partly compensate for
this increase in node parallelism, and keep the peak processing throughput at
reasonable levels.

Change-Id: If4753f27069c45297271d11777361a7454f43278
---
M hieradata/role/common/mediawiki/jobrunner.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/09/259309/1

diff --git a/hieradata/role/common/mediawiki/jobrunner.yaml 
b/hieradata/role/common/mediawiki/jobrunner.yaml
index 0763d0f..84d964c 100644
--- a/hieradata/role/common/mediawiki/jobrunner.yaml
+++ b/hieradata/role/common/mediawiki/jobrunner.yaml
@@ -4,7 +4,7 @@
   - perf-roots
 mediawiki::jobrunner::runners_basic: 20
 mediawiki::jobrunner::runners_gwt: 1
-mediawiki::jobrunner::runners_restbase: 4
+mediawiki::jobrunner::runners_restbase: 3
 mediawiki::jobrunner::runners_translate: 1
 mediawiki::jobrunner::runners_upload: 7
 nutcracker::verbosity: "4"

-- 
To view, visit https://gerrit.wikimedia.org/r/259309
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If4753f27069c45297271d11777361a7454f43278
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 3d55db3 - change (mediawiki...deploy)

2015-12-15 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/259395

Change subject: Update restbase to 3d55db3
..

Update restbase to 3d55db3

List of changes:
56a7145 Override cache-control headers from the pageviews backend
d80ace9 Misc: Improve body_only users logging and sync revision update on 
blacklisted titles

Change-Id: If30df1f0cbbb0784d7e5f1f934114d1328926226
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/95/259395/1

diff --git a/restbase b/restbase
index 463dc17..3d55db3 16
--- a/restbase
+++ b/restbase
-Subproject commit 463dc1743da47b2a6f3534cf7c9f16e263578217
+Subproject commit 3d55db3229cde6b259359d0776326df9080e1810

-- 
To view, visit https://gerrit.wikimedia.org/r/259395
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If30df1f0cbbb0784d7e5f1f934114d1328926226
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 3d55db3 - change (mediawiki...deploy)

2015-12-15 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to 3d55db3
..


Update restbase to 3d55db3

List of changes:
56a7145 Override cache-control headers from the pageviews backend
d80ace9 Misc: Improve body_only users logging and sync revision update on 
blacklisted titles

Change-Id: If30df1f0cbbb0784d7e5f1f934114d1328926226
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/restbase b/restbase
index 463dc17..3d55db3 16
--- a/restbase
+++ b/restbase
-Subproject commit 463dc1743da47b2a6f3534cf7c9f16e263578217
+Subproject commit 3d55db3229cde6b259359d0776326df9080e1810

-- 
To view, visit https://gerrit.wikimedia.org/r/259395
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If30df1f0cbbb0784d7e5f1f934114d1328926226
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 
Gerrit-Reviewer: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Log more detail on some error responses - change (mediawiki...mathoid)

2015-12-14 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/259150

Change subject: Log more detail on some error responses
..

Log more detail on some error responses

We are currently trying to figure out the reason for the high rate of 4xx
responses recently, but don't have good information on the incoming requests.

This patch logs more detail on failed responses with a probability of 10%.

Change-Id: Ie2dbba72ebd59745463f30fdcb58ffcf4ce6944b
---
M routes/mathoid.js
1 file changed, 11 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/mathoid 
refs/changes/50/259150/1

diff --git a/routes/mathoid.js b/routes/mathoid.js
index f9faa92..9b95c82 100644
--- a/routes/mathoid.js
+++ b/routes/mathoid.js
@@ -50,7 +50,7 @@
 format + ": true\" to enable " + format + "rendering.");
 }
 
-function handleRequest(res, q, type, outFormat, features) {
+function handleRequest(res, q, type, outFormat, features, req) {
 var sanitizedTex;
 var svg = app.conf.svg && /^svg|json|complete$/.test(outFormat);
 var mml = (type !== "MathML") && /^mml|json|complete$/.test(outFormat);
@@ -65,6 +65,15 @@
 sanitizedTex = feedback.checked || '';
 q = sanitizedTex;
 } else {
+// Log verbose datails of failed requests with a 10% chance
+if (Math.random() > 0.1) {
+feedback.request = {
+url: req.url,
+params: req.params,
+headers: req.headers,
+body: req.body,
+};
+}
 emitError(feedback.error.name + ': ' + feedback.error.message, 
feedback);
 }
 if (app.conf.texvcinfo && outFormat === "texvcinfo") {
@@ -201,7 +210,7 @@
 } else {
 outFormat = "json";
 }
-handleRequest(res, q, type, outFormat, {speakText:speakText});
+handleRequest(res, q, type, outFormat, {speakText:speakText}, req);
 
 });
 

-- 
To view, visit https://gerrit.wikimedia.org/r/259150
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie2dbba72ebd59745463f30fdcb58ffcf4ce6944b
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/mathoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 02b30ed - change (mediawiki...deploy)

2015-12-14 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/259134

Change subject: Update restbase to 02b30ed
..

Update restbase to 02b30ed

List of changes:
f48775d Tests. Step 1: Separate general framework-related tests from RB related 
tests
038dc72 Removed non-needed files
ec6cc50 Add cache headers in pageviews responses
671ddb9 Cache pageview responses for 1 hour
e3e5607 Reduce cache timeout to 3600s
93f949f Temporarily disable sub-request id check until failure is investigated
53f2ed7 Clean up key_value and key_rev_value option processing

Change-Id: Ibcac3cc0d63a52a756d17a873841b5e7522c3eac
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/34/259134/1

diff --git a/restbase b/restbase
index b2a70c2..02b30ed 16
--- a/restbase
+++ b/restbase
-Subproject commit b2a70c2c1b83da2ac8eedb8fd61a1f722d6e757c
+Subproject commit 02b30edacdc59279aaf2826f4701fe013ac6b7da

-- 
To view, visit https://gerrit.wikimedia.org/r/259134
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibcac3cc0d63a52a756d17a873841b5e7522c3eac
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Make /api/rest_v1/ work for test.wikipedia.org - change (operations/puppet)

2015-12-11 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/258498

Change subject: Make /api/rest_v1/ work for test.wikipedia.org
..

Make /api/rest_v1/ work for test.wikipedia.org

RESTBase does not currently work at test.wikipedia.org, as any request to this
domain (or with the X-Wikimedia-Debug header set) is sent to the
test_wikipedia backend. This patch narrows this down to exclude /api/rest_v1/
matches.

Change-Id: I1c1881b8bc4e5d261ae771d6bfffecbfd7e3947e
---
M templates/varnish/text-backend.inc.vcl.erb
1 file changed, 5 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/98/258498/1

diff --git a/templates/varnish/text-backend.inc.vcl.erb 
b/templates/varnish/text-backend.inc.vcl.erb
index 8a9fc55..052a7c2 100644
--- a/templates/varnish/text-backend.inc.vcl.erb
+++ b/templates/varnish/text-backend.inc.vcl.erb
@@ -16,13 +16,13 @@
set req.backend = api;
} else if (req.url ~ "^/w/thumb(_handler)?\.php") {
set req.backend = rendering;
-   } else if (req.url ~ "^/api/rest_v1/") {
+   }
+   
+   if (req.url ~ "^/api/rest_v1/") {
set req.url = "/" + req.http.host + regsub(req.url, 
"^/api/rest_v1/", "/v1/");
set req.backend = restbase_backend;
-   }
-
-
-   if (req.http.X-Wikimedia-Debug == "1") {
+   } else if (req.http.X-Wikimedia-Debug == "1") {
+   // Note: Make sure only MW core traffic is sent to this backend.
set req.backend = test_wikipedia;
}
 

-- 
To view, visit https://gerrit.wikimedia.org/r/258498
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1c1881b8bc4e5d261ae771d6bfffecbfd7e3947e
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 5d61b92 - change (mediawiki...deploy)

2015-12-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/258229

Change subject: Update restbase to 5d61b92
..

Update restbase to 5d61b92

List of changes:
2c1f6a5 JSCS: switch off the new 'else after return' check
91459f7 Add a static robots.txt in the wikimedia config
79a7e62 Add a rerenderBlacklist option for the parsoid module

Change-Id: I1eca9050db4047720b0fabadb51fc60140b03138
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/29/258229/1

diff --git a/restbase b/restbase
index 6092ac7..5d61b92 16
--- a/restbase
+++ b/restbase
-Subproject commit 6092ac754485f22985854ab136b513da830425c8
+Subproject commit 5d61b926a6f04588c211d1874b6d959fdf30b4e0

-- 
To view, visit https://gerrit.wikimedia.org/r/258229
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1eca9050db4047720b0fabadb51fc60140b03138
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Minor / restbase config: Explicitly set the return status fo... - change (operations/puppet)

2015-12-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/258356

Change subject: Minor / restbase config: Explicitly set the return status for 
robots.txt
..

Minor / restbase config: Explicitly set the return status for robots.txt

Change-Id: Id9ec0918a1b1015cc74ad4745218609d73ec6048
---
M modules/restbase/templates/config.labs.yaml.erb
M modules/restbase/templates/config.yaml.erb
2 files changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/56/258356/1

diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index dae7d2b..45730c6 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -92,6 +92,7 @@
 x-request-handler:
   - static:
   return:
+status: 200
 headers:
   content-type: text/plain
 body: |
diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 007655e..4f2cb6b 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -835,6 +835,7 @@
 x-request-handler:
   - static:
   return:
+status: 200
 headers:
   content-type: text/plain
 body: |

-- 
To view, visit https://gerrit.wikimedia.org/r/258356
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id9ec0918a1b1015cc74ad4745218609d73ec6048
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 9657c4e - change (mediawiki...deploy)

2015-12-10 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to 9657c4e
..


Update restbase to 9657c4e

List of changes:
17ff82d Minor: Fix status for robots.txt
5df59fe Add some more blacklist entries

Change-Id: I85d30b5c0dab44fd0149afe8b82c40cdf803cb30
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/restbase b/restbase
index 5d61b92..9657c4e 16
--- a/restbase
+++ b/restbase
-Subproject commit 5d61b926a6f04588c211d1874b6d959fdf30b4e0
+Subproject commit 9657c4e3b4542723aad6c86480a349c5c90f903e

-- 
To view, visit https://gerrit.wikimedia.org/r/258383
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I85d30b5c0dab44fd0149afe8b82c40cdf803cb30
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 
Gerrit-Reviewer: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 9657c4e - change (mediawiki...deploy)

2015-12-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/258383

Change subject: Update restbase to 9657c4e
..

Update restbase to 9657c4e

List of changes:
17ff82d Minor: Fix status for robots.txt
5df59fe Add some more blacklist entries

Change-Id: I85d30b5c0dab44fd0149afe8b82c40cdf803cb30
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/83/258383/1

diff --git a/restbase b/restbase
index 5d61b92..9657c4e 16
--- a/restbase
+++ b/restbase
-Subproject commit 5d61b926a6f04588c211d1874b6d959fdf30b4e0
+Subproject commit 9657c4e3b4542723aad6c86480a349c5c90f903e

-- 
To view, visit https://gerrit.wikimedia.org/r/258383
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I85d30b5c0dab44fd0149afe8b82c40cdf803cb30
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to b2a70c2 - change (mediawiki...deploy)

2015-12-10 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to b2a70c2
..


Update restbase to b2a70c2

List of changes:
b2a70c2 Add another title to blacklist, per Subbu's request

Change-Id: I1338522fcce18411ba3120c8df1cd5022cdb2834
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/restbase b/restbase
index 9657c4e..b2a70c2 16
--- a/restbase
+++ b/restbase
-Subproject commit 9657c4e3b4542723aad6c86480a349c5c90f903e
+Subproject commit b2a70c2c1b83da2ac8eedb8fd61a1f722d6e757c

-- 
To view, visit https://gerrit.wikimedia.org/r/258424
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I1338522fcce18411ba3120c8df1cd5022cdb2834
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 
Gerrit-Reviewer: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to b2a70c2 - change (mediawiki...deploy)

2015-12-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/258424

Change subject: Update restbase to b2a70c2
..

Update restbase to b2a70c2

List of changes:
b2a70c2 Add another title to blacklist, per Subbu's request

Change-Id: I1338522fcce18411ba3120c8df1cd5022cdb2834
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/24/258424/1

diff --git a/restbase b/restbase
index 9657c4e..b2a70c2 16
--- a/restbase
+++ b/restbase
-Subproject commit 9657c4e3b4542723aad6c86480a349c5c90f903e
+Subproject commit b2a70c2c1b83da2ac8eedb8fd61a1f722d6e757c

-- 
To view, visit https://gerrit.wikimedia.org/r/258424
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1338522fcce18411ba3120c8df1cd5022cdb2834
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add a robots.txt to disallow indexing of API content - change (operations/puppet)

2015-12-09 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257922

Change subject: Add a robots.txt to disallow indexing of API content
..

Add a robots.txt to disallow indexing of API content

But, make some limited effort to get the docs indexed.

Bug: T119786

Change-Id: I338a15398cb6583d417e6869acfe428f9fa3aa5e
---
M modules/restbase/templates/config.labs.yaml.erb
M modules/restbase/templates/config.yaml.erb
2 files changed, 24 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/22/257922/1

diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index 0b92d0d..25a6f18 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -86,6 +86,18 @@
 # global domain
 /{domain:wikimedia.org}: *wikimedia.org
 
+# A robots.txt to make sure that the content isn't indexed.
+/robots.txt:
+  get:
+x-request-handler:
+  - static:
+  return:
+headers:
+  content-type: text/plain
+body: |
+  User-agent: *
+  Allow: /*/v1/?doc
+  Disallow: /
 
 
 # Finally, a standard service-runner config.
diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 0baff58..007655e 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -829,6 +829,18 @@
 # global domain
 /{domain:wikimedia.org}: *wikimedia.org
 
+# A robots.txt to make sure that the content isn't indexed.
+/robots.txt:
+  get:
+x-request-handler:
+  - static:
+  return:
+headers:
+  content-type: text/plain
+body: |
+  User-agent: *
+  Allow: /*/v1/?doc
+  Disallow: /
 
 
 # Finally, a standard service-runner config.

-- 
To view, visit https://gerrit.wikimedia.org/r/257922
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I338a15398cb6583d417e6869acfe428f9fa3aa5e
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Fix the api URI template for labs - change (operations/puppet)

2015-12-09 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257965

Change subject: Fix the api URI template for labs
..

Fix the api URI template for labs

Change-Id: I8330084ae8352c675d0ae71dc27d2e1d965124f3
---
M modules/restbase/templates/config.labs.yaml.erb
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/65/257965/1

diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index 0b92d0d..e2b203d 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -23,7 +23,7 @@
   parsoid:
 host: <%= @parsoid_uri %>
   action:
-apiUriTemplate: http://api.svc.eqiad.wmnet/w/api.php
+apiUriTemplate: "{{'http://{domain}/w/api.php'}}"
   graphoid:
 host: <%= @graphoid_uri %>
   mathoid:

-- 
To view, visit https://gerrit.wikimedia.org/r/257965
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8330084ae8352c675d0ae71dc27d2e1d965124f3
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 6092ac7 - change (mediawiki...deploy)

2015-12-08 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to 6092ac7
..


Update restbase to 6092ac7

List of changes:
d95f636 Don't modify the original spec
e30b20b Define /sys/ as api roots to scope metrics

Change-Id: I8a8bff5f7d5a00c0eb9420f8db386a7ecf00496e
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/restbase b/restbase
index 8ee5045..6092ac7 16
--- a/restbase
+++ b/restbase
-Subproject commit 8ee504542fb2e9d01e70df96d0edcd18fd9da62a
+Subproject commit 6092ac754485f22985854ab136b513da830425c8

-- 
To view, visit https://gerrit.wikimedia.org/r/257685
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I8a8bff5f7d5a00c0eb9420f8db386a7ecf00496e
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 
Gerrit-Reviewer: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Share the RESTBase config template between production and labs - change (operations/puppet)

2015-12-08 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257696

Change subject: Share the RESTBase config template between production and labs
..

Share the RESTBase config template between production and labs

The labs restbase hosts are going to continue using puppet for config
management until scap3 matures. Puppet has recently been re-enabled on
deployment-restbase01, which broke restbase there as no valid config was
supplied.

Thanks to the recent refactor reducing the config to variables only we should
have no trouble sharing the same config template between production & labs.
This patch makes it so.

Change-Id: I81b83c9fb908e69f623224359f7a1be693e63af8
---
M hieradata/labs/deployment-prep/common.yaml
D modules/restbase/templates/config.labs.yaml.erb
2 files changed, 0 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/96/257696/1

diff --git a/hieradata/labs/deployment-prep/common.yaml 
b/hieradata/labs/deployment-prep/common.yaml
index 7c899f0..ca699df 100644
--- a/hieradata/labs/deployment-prep/common.yaml
+++ b/hieradata/labs/deployment-prep/common.yaml
@@ -65,7 +65,6 @@
 restbase::seeds:
   - 10.68.17.227
   - 10.68.17.189
-restbase::config_template: restbase/config.labs.yaml.erb
 restbase::parsoid_uri: 
http://deployment-parsoid05.deployment-prep.eqiad.wmflabs:8000
 restbase::statsd_host: labmon1001.eqiad.wmnet
 restbase::logstash_host: deployment-logstash2.deployment-prep.eqiad.wmflabs
diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
deleted file mode 100644
index 622f72b..000
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ /dev/null
@@ -1,3 +0,0 @@
-# Current labs testing is using scap3, so this is entirely unused. If we start
-# using this again, we should really share a config with production.
-# -- gwicke Mon Dec  7 12:00:43 PST 2015

-- 
To view, visit https://gerrit.wikimedia.org/r/257696
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I81b83c9fb908e69f623224359f7a1be693e63af8
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 6092ac7 - change (mediawiki...deploy)

2015-12-08 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257685

Change subject: Update restbase to 6092ac7
..

Update restbase to 6092ac7

List of changes:
d95f636 Don't modify the original spec
e30b20b Define /sys/ as api roots to scope metrics

Change-Id: I8a8bff5f7d5a00c0eb9420f8db386a7ecf00496e
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/85/257685/1

diff --git a/restbase b/restbase
index 8ee5045..6092ac7 16
--- a/restbase
+++ b/restbase
-Subproject commit 8ee504542fb2e9d01e70df96d0edcd18fd9da62a
+Subproject commit 6092ac754485f22985854ab136b513da830425c8

-- 
To view, visit https://gerrit.wikimedia.org/r/257685
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8a8bff5f7d5a00c0eb9420f8db386a7ecf00496e
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add a hack to work around Parsoid's fatal exiting logger exp... - change (mediawiki...parsoid)

2015-12-08 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257771

Change subject: Add a hack to work around Parsoid's fatal exiting logger 
expectation
..

Add a hack to work around Parsoid's fatal exiting logger expectation

Change-Id: I1eb9300af7da54e18590f99b2f1d275f6afa8dd2
---
M lib/index.js
1 file changed, 24 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/71/257771/1

diff --git a/lib/index.js b/lib/index.js
index 5186ac0..9ad3409 100644
--- a/lib/index.js
+++ b/lib/index.js
@@ -145,5 +145,28 @@
  */
 Parsoid.apiServiceWorker = function apiServiceWorker(options) {
var parsoidConfig = new ParsoidConfig(null, options.config);
-   return ParsoidService.init(parsoidConfig, options.logger);
+
+   // Hack: Parsoid currently relies on the logger to exit after logging a
+   // fatal. Emulate this with a small wrapper.
+   // TODO: Perhaps ask the logger to return a promise & exit when that
+   // fires? Do the timeout thing here in a delayedExit utility in Parsoid?
+   var hackyLogger = {
+   log: function() {
+   // Copy arguments to preserve optimization
+   var args = new Array(arguments.length);
+   for (var i = 0; i < arguments.length; i++) {
+   args[i] = arguments[i];
+   }
+   options.logger.log.apply(options.logger, args);
+   if (/^fatal\b/.test(args[0])) {
+   // Hack for Parsoid: Delay a bit to give the 
logger time to do
+   // its thing, then exit.
+   setTimeout(function() {
+   process.exit(1);
+   }, 1000);
+   }
+   }
+   };
+
+   return ParsoidService.init(parsoidConfig, hackyLogger);
 };

-- 
To view, visit https://gerrit.wikimedia.org/r/257771
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1eb9300af7da54e18590f99b2f1d275f6afa8dd2
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Reinstate separate labs config - change (operations/puppet)

2015-12-08 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257743

Change subject: Reinstate separate labs config
..

Reinstate separate labs config

Sadly, sharing the config between labs & production is complicated by the
difference in domains configured being large. We'll have to discuss whether
storing this in hiera is appropriate or not.

In the meantime, this patch aims to fix restbase in labs by updating the list
of domains in place until a better solution can be found.

Change-Id: I8b637dd887f501d89cf7db1093070ff9d2461598
---
M modules/restbase/templates/config.labs.yaml.erb
1 file changed, 147 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/43/257743/1

diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index 622f72b..cdf5527 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -1,3 +1,147 @@
-# Current labs testing is using scap3, so this is entirely unused. If we start
-# using this again, we should really share a config with production.
-# -- gwicke Mon Dec  7 12:00:43 PST 2015
+# RESTBase wikimedia config
+
+# First, load & parametrize project templates. These are referenced / shared
+# between domains in the root_spec further down.
+default_project: _project
+  x-modules:
+/:
+  - path: projects/wmf_default.yaml
+options: _options
+  table:
+hosts: [<%= Array(@seeds).join(',') %>]
+keyspace: system
+localDc: <%= @cassandra_localDc %>
+datacenters: [<%= Array(@cassandra_datacenters).join(',') %>]
+username: <%= @cassandra_user %>
+password: <%= @cassandra_password %>
+defaultConsistency: <%= @cassandra_defaultConsistency %>
+storage_groups:
+  - name: phase0.group.local
+domains:
+  - /^(?:test.*\.wiki.*\.org|www.mediawiki.org)$/
+  - name: wikipedia.group.local
+domains: /\.wikipedia\.org$/
+  - name: wikimedia.group.local
+domains: /\.wikimedia\.org$/
+  - name: wiktionary.group.local
+domains: /\.wiktionary\.org$/
+  - name: wikibooks.group.local
+domains: /\.wikibooks\.org$/
+  - name: wikisource.group.local
+domains: /\.wikisource\.org$/
+  - name: wikiquote.group.local
+domains: /\.wikiquote\.org$/
+  - name: wikinews.group.local
+domains: /\.wikinews\.org$/
+  - name: wikiversity.group.local
+domains: /\.wikiversity\.org$/
+  - name: wikivoyage.group.local
+domains: /\.wikivoyage\.org$/
+  - name: globaldomain.group.local
+domains: /^wikimedia\.org$/
+  # Catch-all group
+  - name: default.group.local
+domains: /./
+
+  parsoid:
+host: <%= @parsoid_uri %>
+  action:
+apiUriTemplate: http://api.svc.eqiad.wmnet/w/api.php
+  graphoid:
+host: <%= @graphoid_uri %>
+  mathoid:
+host: <%= @mathoid_uri %>
+# 10 days Varnish caching, one day client-side
+cache-control: s-maxage=864000, max-age=86400
+  mobileapps:
+host: <%= @mobileapps_uri %>
+
+# A different project template, sharing configuration options.
+wikimedia.org: 
+  x-modules:
+/:
+  - path: projects/wikimedia.org.yaml
+options:
+<<: *default_options
+pageviews:
+  host: <%= @aqs_uri %>
+
+
+# Swagger spec root.
+root_spec: _spec
+  title: "The RESTBase root"
+  # Some more general RESTBase info
+  paths:
+# list taken from Parsoid's beta config
+/{domain:aa.wikipedia.beta.wmflabs.org}: *default_project
+/{domain:ar.wikipedia.beta.wmflabs.org}: *default_project
+/{domain:ca.wikipedia.beta.wmflabs.org}: *default_project
+/{domain:commons.wikimedia.beta.wmflabs.org}: *default_project
+/{domain:deployment.wikimedia.beta.wmflabs.org}: *default_project
+/{domain:de.wikipedia.beta.wmflabs.org}: *default_project
+/{domain:en-rtl.wikipedia.beta.wmflabs.org}: *default_project
+/{domain:en.wikibooks.beta.wmflabs.org}: *default_project
+/{domain:en.wikinews.beta.wmflabs.org}: *default_project
+/{domain:en.wikipedia.beta.wmflabs.org}: *default_project
+/{domain:en.wikiquote.beta.wmflabs.org}: *default_project
+/{domain:en.wikisource.beta.wmflabs.org}: *default_project
+/{domain:en.wikitionary.beta.wmflabs.org}: *default_project
+/{domain:en.wikiversity.beta.wmflabs.org}: *default_project
+/{domain:en.wikivoyage.beta.wmflabs.org}: *default_project
+

[MediaWiki-commits] [Gerrit] Update restbase to 87f0666 - change (mediawiki...deploy)

2015-12-07 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257531

Change subject: Update restbase to 87f0666
..

Update restbase to 87f0666

List of changes:
2281217 Make mathoid responses cacheable
0cc6512 Require s-maxage in x-ample, and set cache headers in test config
dbbdc23 Trivial: Export actual entry point & release v0.9.1
335e6c6 Specs: Removed test module from the default project
9f3914c Serving the css source map correctly
7b7883f Update dependencies
1421ea4 Bump cassandra and sqlite modules versions
567154b Add a pageviews_proxy.yaml config
f411a51 Fix module & subtree sharing
xxx Update node module dependencies

Change-Id: Ifadb465e294ad6c6d5c0f0272185d6238492e7d6
---
D node_modules/ajv-i18n/LICENSE
D node_modules/ajv-i18n/localize/de/index.js
D node_modules/ajv-i18n/localize/en/index.js
D node_modules/ajv-i18n/localize/hu/index.js
D node_modules/ajv-i18n/localize/index.js
D node_modules/ajv-i18n/localize/it/index.js
D node_modules/ajv-i18n/localize/ja/index.js
D node_modules/ajv-i18n/localize/localize.jst
D node_modules/ajv-i18n/localize/pl/index.js
D node_modules/ajv-i18n/localize/ru/index.js
D node_modules/ajv-i18n/package.json
A node_modules/ajv/.tonic_example.js
M node_modules/ajv/lib/ajv.js
A node_modules/ajv/lib/async.js
M node_modules/ajv/lib/compile/formats.js
M node_modules/ajv/lib/compile/index.js
A node_modules/ajv/lib/compile/macro.js
M node_modules/ajv/lib/compile/resolve.js
M node_modules/ajv/lib/compile/rules.js
M node_modules/ajv/lib/compile/util.js
A node_modules/ajv/lib/dot/custom.def
M node_modules/ajv/lib/dot/definitions.def
M node_modules/ajv/lib/dot/dependencies.jst
M node_modules/ajv/lib/dot/format.jst
A node_modules/ajv/lib/dot/missing.def
M node_modules/ajv/lib/dot/properties.jst
M node_modules/ajv/lib/dot/ref.jst
M node_modules/ajv/lib/dot/required.jst
M node_modules/ajv/lib/dot/validate.jst
M node_modules/ajv/lib/dotjs/anyOf.js
M node_modules/ajv/lib/dotjs/dependencies.js
M node_modules/ajv/lib/dotjs/enum.js
M node_modules/ajv/lib/dotjs/format.js
M node_modules/ajv/lib/dotjs/items.js
M node_modules/ajv/lib/dotjs/maxItems.js
M node_modules/ajv/lib/dotjs/maxLength.js
M node_modules/ajv/lib/dotjs/maxProperties.js
M node_modules/ajv/lib/dotjs/maximum.js
M node_modules/ajv/lib/dotjs/minItems.js
M node_modules/ajv/lib/dotjs/minLength.js
M node_modules/ajv/lib/dotjs/minProperties.js
M node_modules/ajv/lib/dotjs/minimum.js
M node_modules/ajv/lib/dotjs/multipleOf.js
M node_modules/ajv/lib/dotjs/not.js
M node_modules/ajv/lib/dotjs/oneOf.js
M node_modules/ajv/lib/dotjs/pattern.js
M node_modules/ajv/lib/dotjs/properties.js
M node_modules/ajv/lib/dotjs/ref.js
M node_modules/ajv/lib/dotjs/required.js
M node_modules/ajv/lib/dotjs/uniqueItems.js
M node_modules/ajv/lib/dotjs/validate.js
A node_modules/ajv/lib/keyword.js
A node_modules/ajv/lib/v5.js
M node_modules/ajv/package.json
M node_modules/busboy/package.json
M node_modules/core-js/package.json
R node_modules/extend/.eslintrc
R node_modules/extend/.jscs.json
R node_modules/extend/.npmignore
R node_modules/extend/.travis.yml
R node_modules/extend/LICENSE
R node_modules/extend/component.json
R node_modules/extend/index.js
R node_modules/extend/package.json
M node_modules/js-yaml/package.json
M node_modules/jsonwebtoken/package.json
D node_modules/preq/node_modules/request/node_modules/extend/.eslintrc
D node_modules/preq/node_modules/request/node_modules/extend/.jscs.json
D node_modules/preq/node_modules/request/node_modules/extend/.npmignore
D node_modules/preq/node_modules/request/node_modules/extend/.travis.yml
D node_modules/preq/node_modules/request/node_modules/extend/LICENSE
D node_modules/preq/node_modules/request/node_modules/extend/component.json
D node_modules/preq/node_modules/request/node_modules/extend/index.js
M node_modules/preq/package.json
A node_modules/restbase-mod-table-cassandra/.jscs.json
M node_modules/restbase-mod-table-cassandra/index.js
M node_modules/restbase-mod-table-cassandra/lib/db.js
M node_modules/restbase-mod-table-cassandra/lib/dbutils.js
M node_modules/restbase-mod-table-cassandra/lib/index.js
M node_modules/restbase-mod-table-cassandra/lib/revisionPolicy.js
M node_modules/restbase-mod-table-cassandra/lib/schemaMigration.js
M node_modules/restbase-mod-table-cassandra/lib/secondaryIndexes.js
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/package.json
D node_modules/restbase-mod-table-cassandra/node_modules/extend/.jscs.json
D node_modules/restbase-mod-table-cassandra/node_modules/extend/.npmignore
D node_modules/restbase-mod-table-cassandra/node_modules/extend/.travis.yml
D node_modules/restbase-mod-table-cassandra/node_modules/extend/LICENSE
D node_modules/restbase-mod-table-cassandra/node_modules/extend/component.json
D

[MediaWiki-commits] [Gerrit] Update restbase to 87f0666 - change (mediawiki...deploy)

2015-12-07 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to 87f0666
..


Update restbase to 87f0666

List of changes:
2281217 Make mathoid responses cacheable
0cc6512 Require s-maxage in x-ample, and set cache headers in test config
dbbdc23 Trivial: Export actual entry point & release v0.9.1
335e6c6 Specs: Removed test module from the default project
9f3914c Serving the css source map correctly
7b7883f Update dependencies
1421ea4 Bump cassandra and sqlite modules versions
567154b Add a pageviews_proxy.yaml config
f411a51 Fix module & subtree sharing
xxx Update node module dependencies

Change-Id: Ifadb465e294ad6c6d5c0f0272185d6238492e7d6
---
D node_modules/ajv-i18n/LICENSE
D node_modules/ajv-i18n/localize/de/index.js
D node_modules/ajv-i18n/localize/en/index.js
D node_modules/ajv-i18n/localize/hu/index.js
D node_modules/ajv-i18n/localize/index.js
D node_modules/ajv-i18n/localize/it/index.js
D node_modules/ajv-i18n/localize/ja/index.js
D node_modules/ajv-i18n/localize/localize.jst
D node_modules/ajv-i18n/localize/pl/index.js
D node_modules/ajv-i18n/localize/ru/index.js
D node_modules/ajv-i18n/package.json
A node_modules/ajv/.tonic_example.js
M node_modules/ajv/lib/ajv.js
A node_modules/ajv/lib/async.js
M node_modules/ajv/lib/compile/formats.js
M node_modules/ajv/lib/compile/index.js
A node_modules/ajv/lib/compile/macro.js
M node_modules/ajv/lib/compile/resolve.js
M node_modules/ajv/lib/compile/rules.js
M node_modules/ajv/lib/compile/util.js
A node_modules/ajv/lib/dot/custom.def
M node_modules/ajv/lib/dot/definitions.def
M node_modules/ajv/lib/dot/dependencies.jst
M node_modules/ajv/lib/dot/format.jst
A node_modules/ajv/lib/dot/missing.def
M node_modules/ajv/lib/dot/properties.jst
M node_modules/ajv/lib/dot/ref.jst
M node_modules/ajv/lib/dot/required.jst
M node_modules/ajv/lib/dot/validate.jst
M node_modules/ajv/lib/dotjs/anyOf.js
M node_modules/ajv/lib/dotjs/dependencies.js
M node_modules/ajv/lib/dotjs/enum.js
M node_modules/ajv/lib/dotjs/format.js
M node_modules/ajv/lib/dotjs/items.js
M node_modules/ajv/lib/dotjs/maxItems.js
M node_modules/ajv/lib/dotjs/maxLength.js
M node_modules/ajv/lib/dotjs/maxProperties.js
M node_modules/ajv/lib/dotjs/maximum.js
M node_modules/ajv/lib/dotjs/minItems.js
M node_modules/ajv/lib/dotjs/minLength.js
M node_modules/ajv/lib/dotjs/minProperties.js
M node_modules/ajv/lib/dotjs/minimum.js
M node_modules/ajv/lib/dotjs/multipleOf.js
M node_modules/ajv/lib/dotjs/not.js
M node_modules/ajv/lib/dotjs/oneOf.js
M node_modules/ajv/lib/dotjs/pattern.js
M node_modules/ajv/lib/dotjs/properties.js
M node_modules/ajv/lib/dotjs/ref.js
M node_modules/ajv/lib/dotjs/required.js
M node_modules/ajv/lib/dotjs/uniqueItems.js
M node_modules/ajv/lib/dotjs/validate.js
A node_modules/ajv/lib/keyword.js
A node_modules/ajv/lib/v5.js
M node_modules/ajv/package.json
M node_modules/busboy/package.json
M node_modules/core-js/package.json
R node_modules/extend/.eslintrc
R node_modules/extend/.jscs.json
R node_modules/extend/.npmignore
R node_modules/extend/.travis.yml
R node_modules/extend/LICENSE
R node_modules/extend/component.json
R node_modules/extend/index.js
R node_modules/extend/package.json
M node_modules/js-yaml/package.json
M node_modules/jsonwebtoken/package.json
D node_modules/preq/node_modules/request/node_modules/extend/.eslintrc
D node_modules/preq/node_modules/request/node_modules/extend/.jscs.json
D node_modules/preq/node_modules/request/node_modules/extend/.npmignore
D node_modules/preq/node_modules/request/node_modules/extend/.travis.yml
D node_modules/preq/node_modules/request/node_modules/extend/LICENSE
D node_modules/preq/node_modules/request/node_modules/extend/component.json
D node_modules/preq/node_modules/request/node_modules/extend/index.js
M node_modules/preq/package.json
A node_modules/restbase-mod-table-cassandra/.jscs.json
M node_modules/restbase-mod-table-cassandra/index.js
M node_modules/restbase-mod-table-cassandra/lib/db.js
M node_modules/restbase-mod-table-cassandra/lib/dbutils.js
M node_modules/restbase-mod-table-cassandra/lib/index.js
M node_modules/restbase-mod-table-cassandra/lib/revisionPolicy.js
M node_modules/restbase-mod-table-cassandra/lib/schemaMigration.js
M node_modules/restbase-mod-table-cassandra/lib/secondaryIndexes.js
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/package.json
D node_modules/restbase-mod-table-cassandra/node_modules/extend/.jscs.json
D node_modules/restbase-mod-table-cassandra/node_modules/extend/.npmignore
D node_modules/restbase-mod-table-cassandra/node_modules/extend/.travis.yml
D node_modules/restbase-mod-table-cassandra/node_modules/extend/LICENSE
D node_modules/restbase-mod-table-cassandra/node_modules/extend/component.json
D node_modules/restbase-mod-table-cassandra/node_modules/extend/index.js
D

[MediaWiki-commits] [Gerrit] Update RESTBase configs for RESTBase v0.9.1 - change (operations/puppet)

2015-12-07 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257408

Change subject: Update RESTBase configs for RESTBase v0.9.1
..

Update RESTBase configs for RESTBase v0.9.1

RESTBase 0.9 comes with a major cleanup in config handling. This moves most
service configurations out of the top-level config, and instead uses spec
modules to configure the service for each project.

Since this is a large change that we can't really test outside of puppet,
we'll need to thoroughly test this in staging before carefully rolling it out
in production. We have tested a similar configuration change in labs, but that
is not using the exact same template, and could thus differ in behavior from
production.

Change-Id: If4fc33c7bcf42cf9cde8d31b5bb9b6351142c59a
WARNING: DO NOT MERGE UNTIL PUPPET IS DISABLED ON THE PRODUCTION RB CLUSTER!
---
M modules/restbase/templates/config.labs.yaml.erb
M modules/restbase/templates/config.yaml.erb
2 files changed, 808 insertions(+), 1,321 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/08/257408/1

diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index f836a76..622f72b 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -1,325 +1,3 @@
-# RESTBase config
-
-info:
-  name: restbase
-
-
-
-# Swagger spec templates, referenced using yaml references in the spec section
-# below.
-templates:
-
-  wmf-content-1.0.0: /content/1.0.0
-swagger: '2.0'
-# swagger options, overriding the shared ones from the merged specs (?)
-info: /content-info/1.0.0
-  version: 1.0.0-beta
-  title: Wikimedia REST API
-  description: >
-  This API aims to provide coherent and low-latency access to
-  Wikimedia content and services. It is currently in beta testing, so
-  things aren't completely locked down yet. Each entry point has
-  explicit stability markers to inform you about development status
-  and change policy, according to [our API version
-  policy](https://www.mediawiki.org/wiki/API_versioning).
-
-  ### High-volume access
-- As a general rule, don't perform more than 200 requests/s to
-  this API.
-- Set a unique `User-Agent` header that allows us to contact you
-  quickly. Email addresses or URLs of contact pages work well.
-- Consider using our [HTML
-  dumps](https://phabricator.wikimedia.org/T17017) once they
-  become available.
-
-  termsOfService: https://wikimediafoundation.org/wiki/Terms_of_Use
-  contact:
-name: the Wikimedia Services team
-url: http://mediawiki.org/wiki/RESTBase
-  license:
-name: Apache2
-url: http://www.apache.org/licenses/LICENSE-2.0
-
-securityDefinitions: /content-security/1.0.0
-  mediawiki_auth:
-description: Checks permissions using MW api
-type: apiKey
-in: header
-name: cookie
-x-internal-request-whitelist:
-  - /^https?:\/\/.+wmflabs.org.*\/w\/api\.php$/
-  - <%= @parsoid_uri %>
-  - <%= @graphoid_uri %>
-  - <%= @mobileapps_uri %>
-  - <%= @mathoid_uri %>
-  - <%= @aqs_uri %>
-  header_match:
-description: Checks client ip against one of the predefined whitelists
-x-error-message: This client is not allowed to use the endpoint
-x-whitelists:
-  internal:
-- /^(?::::)?(?:10|127)\./
-
-# Override the base path for host-based (proxied) requests. In our case,
-# we proxy https://{domain}/api/rest_v1/ to the API.
-x-host-basePath: /api/rest_v1
-
-x-modules:
-  - name: mobileapps-public
-path: specs/mediawiki/v1/mobileapps
-type: spec
-options:
-  host: <%= @mobileapps_uri %>
-
-  - name: graphoid
-path: wikimedia/v1/graphoid_v1
-type: spec
-options:
-  host: <%= @graphoid_uri %>
-
-  - name: page-summary
-path: wikimedia/v1/summary_v1.js
-options:
-  # One hour, for now.
-  response_cache-control: 'max-age: 3600, s-maxage: 3600'
-
-  - name: mathoid
-path: specs/media/v1/mathoid
-type: spec
-options:
-  host: <%= @mathoid_uri %>
-
-x-subspecs:
-  - mediawiki/v1/content
-
-
-  wmf-sys-1.0.0: /sys/1.0.0
-info:
-  title: Default MediaWiki sys API module
-  version: 1.0.0
-paths:
-  /{module:table}: /sys/table
-x-modules:
-  # There can be multiple modules too per stanza, as long as the
-  # exported symbols don't conflict. The operationIds from the spec
-  # will be resolved against all of the modules.
-  - name: restbase-mod-table-cassandra
-

[MediaWiki-commits] [Gerrit] Export a service-runner compatible entry point - change (mediawiki...parsoid)

2015-12-06 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/257213

Change subject: Export a service-runner compatible entry point
..

Export a service-runner compatible entry point

This patch adds a service-runner compatible entry point to Parsoid's public
mudule API. Using this entry point, it is possible to run Parsoid using
service-runner. Among other things, this also allows us to run Parsoid,
RESTBase and other services in a single node process.

Change-Id: Ia057823d0173dcff090c1afef329b4cf64de8d73
---
M lib/config/ParsoidConfig.js
M lib/index.js
2 files changed, 18 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/13/257213/1

diff --git a/lib/config/ParsoidConfig.js b/lib/config/ParsoidConfig.js
index 110ccaa..26ee8a3 100644
--- a/lib/config/ParsoidConfig.js
+++ b/lib/config/ParsoidConfig.js
@@ -91,6 +91,7 @@
  * @param {Object} options Any options we want to set over the defaults. Will 
not overwrite things set by the localSettings.setup function. See the class 
properties for more information.
  */
 function ParsoidConfig(localSettings, options) {
+   var self = this;
this.mwApiMap = new Map();
this.reverseMwApiMap = new Map();
this.mwApiRegexp = "";
@@ -106,6 +107,12 @@
// This happily overwrites inherited properties.
if (options) {
Object.assign(this, options);
+   // Call setMwApi for each specified API.
+   if (Array.isArray(options.mwApis)) {
+   options.mwApis.forEach(function(api) {
+   self.setMwApi(api);
+   });
+   }
}
 
if (this.loadWMF) {
diff --git a/lib/index.js b/lib/index.js
index 1369874..d580e19 100644
--- a/lib/index.js
+++ b/lib/index.js
@@ -4,6 +4,7 @@
 var json = require('../package.json');
 var parseJs = require('../bin/parse.js');
 var ParsoidConfig = require('./config/ParsoidConfig.js').ParsoidConfig;
+var ParsoidService = require("./api/ParsoidService.js");
 var JsApi = require('./jsapi.js');
 
 /**
@@ -136,3 +137,13 @@
 Object.keys(JsApi).forEach(function(k) {
Parsoid[k] = JsApi[k];
 });
+
+/**
+ * Start an API service worker as part of a service-runner service.
+ * @param {object} options
+ * @return {Promise}
+ */
+Parsoid.apiServiceWorker = function apiServiceWorker(options) {
+   var parsoidConfig = new ParsoidConfig(null, options.config);
+   return ParsoidService.init(parsoidConfig, options.logger);
+}

-- 
To view, visit https://gerrit.wikimedia.org/r/257213
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia057823d0173dcff090c1afef329b4cf64de8d73
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to b3d73a6 - change (mediawiki...deploy)

2015-12-04 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/256989

Change subject: Update restbase to b3d73a6
..

Update restbase to b3d73a6

List of changes:
d7a34b0 WIP: Spec format & router cleanup
d2a3435 Cleaner x-modules & config updates
c34fb6c Move project definitions into sub-specs
c6117d0 More clean-up
ccac420 Update configs to reflect mathoid changes
95020af Fix ETag header in storage miss response
52f3e58 Drop unused sys/mathoid module & small fixes
5ac57cd Protect options.templates from expansion
797c1e5 Bump version to 0.9.0
b922fc2 Remove module caching & cleanup
6049009 Remove stray /page prefix
xxx Update node module dependencies

Change-Id: I056a1ef7d6f1e7ba4fbed146bc65fef13a0bad04
---
R 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/index.js
R 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/license
R 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/routeswitch/node_modules/async/package.json
M node_modules/service-runner/node_modules/yargs/index.js
M node_modules/service-runner/node_modules/yargs/lib/parser.js
M node_modules/service-runner/node_modules/yargs/lib/usage.js
A node_modules/service-runner/node_modules/yargs/locales/id.json
A node_modules/service-runner/node_modules/yargs/locales/ko.json
A node_modules/service-runner/node_modules/yargs/locales/nb.json
A node_modules/service-runner/node_modules/yargs/locales/tr.json
A node_modules/service-runner/node_modules/yargs/node_modules/camelcase/index.js
C node_modules/service-runner/node_modules/yargs/node_modules/camelcase/license
C 
node_modules/service-runner/node_modules/yargs/node_modules/camelcase/package.json
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/index.js
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/license
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/code-point-at/index.js
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/code-point-at/license
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/code-point-at/package.json
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/is-fullwidth-code-point/index.js
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/is-fullwidth-code-point/license
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/is-fullwidth-code-point/package.json
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/package.json
M 
node_modules/service-runner/node_modules/yargs/node_modules/window-size/package.json
M node_modules/service-runner/node_modules/yargs/package.json
M restbase
27 files changed, 253 insertions(+), 36 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/89/256989/1

diff --git a/node_modules/camelcase/index.js 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/index.js
similarity index 100%
rename from node_modules/camelcase/index.js
rename to 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/index.js
diff --git a/node_modules/camelcase/license 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/license
similarity index 100%
rename from node_modules/camelcase/license
rename to 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/license
diff --git a/node_modules/camelcase/package.json 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/package.json
similarity index 94%
rename from node_modules/camelcase/package.json
rename to 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/package.json
index 895db7c..afb1aec 100644
--- a/node_modules/camelcase/package.json
+++ 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/package.json
@@ -45,7 +45,7 @@
   "homepage":

[MediaWiki-commits] [Gerrit] Update restbase to b3d73a6 - change (mediawiki...deploy)

2015-12-04 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to b3d73a6
..


Update restbase to b3d73a6

List of changes:
d7a34b0 WIP: Spec format & router cleanup
d2a3435 Cleaner x-modules & config updates
c34fb6c Move project definitions into sub-specs
c6117d0 More clean-up
ccac420 Update configs to reflect mathoid changes
95020af Fix ETag header in storage miss response
52f3e58 Drop unused sys/mathoid module & small fixes
5ac57cd Protect options.templates from expansion
797c1e5 Bump version to 0.9.0
b922fc2 Remove module caching & cleanup
6049009 Remove stray /page prefix
xxx Update node module dependencies

Change-Id: I056a1ef7d6f1e7ba4fbed146bc65fef13a0bad04
---
R 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/index.js
R 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/license
R 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/routeswitch/node_modules/async/package.json
M node_modules/service-runner/node_modules/yargs/index.js
M node_modules/service-runner/node_modules/yargs/lib/parser.js
M node_modules/service-runner/node_modules/yargs/lib/usage.js
A node_modules/service-runner/node_modules/yargs/locales/id.json
A node_modules/service-runner/node_modules/yargs/locales/ko.json
A node_modules/service-runner/node_modules/yargs/locales/nb.json
A node_modules/service-runner/node_modules/yargs/locales/tr.json
A node_modules/service-runner/node_modules/yargs/node_modules/camelcase/index.js
C node_modules/service-runner/node_modules/yargs/node_modules/camelcase/license
C 
node_modules/service-runner/node_modules/yargs/node_modules/camelcase/package.json
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/index.js
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/license
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/code-point-at/index.js
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/code-point-at/license
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/code-point-at/package.json
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/is-fullwidth-code-point/index.js
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/is-fullwidth-code-point/license
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/node_modules/is-fullwidth-code-point/package.json
R 
node_modules/service-runner/node_modules/yargs/node_modules/string-width/package.json
M 
node_modules/service-runner/node_modules/yargs/node_modules/window-size/package.json
M node_modules/service-runner/node_modules/yargs/package.json
M restbase
27 files changed, 253 insertions(+), 36 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/node_modules/camelcase/index.js 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/index.js
similarity index 100%
rename from node_modules/camelcase/index.js
rename to 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/index.js
diff --git a/node_modules/camelcase/license 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/license
similarity index 100%
rename from node_modules/camelcase/license
rename to 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/license
diff --git a/node_modules/camelcase/package.json 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/package.json
similarity index 94%
rename from node_modules/camelcase/package.json
rename to 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/package.json
index 895db7c..afb1aec 100644
--- a/node_modules/camelcase/package.json
+++ 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/meow/node_modules/camelcase-keys/node_modules/camelcase/package.json
@@ -45,7 +45,7 @@
   "homepage": "https://github.com/sindresorhus/camelcase;,
   "_id": "camelcase@1.2.1",
   "_shasum":

[MediaWiki-commits] [Gerrit] Update restbase to 2da393c - change (mediawiki...deploy)

2015-11-19 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/254370

Change subject: Update restbase to 2da393c
..

Update restbase to 2da393c

List of changes:
xxx Update node module dependencies

Change-Id: I17e5168cc63285cdfb219479426cd23fc7a8c9ce
---
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
M node_modules/tassembly/browser/tassembly.js
M node_modules/tassembly/browser/tassembly.orig.js
M node_modules/tassembly/package.json
M node_modules/tassembly/tassembly.js
M node_modules/tassembly/test.js
7 files changed, 70 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/70/254370/1

diff --git 
a/node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
 
b/node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
index 76d5799..062a245 100644
--- 
a/node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
+++ 
b/node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
@@ -57,7 +57,7 @@
   "homepage": "https://github.com/caolan/async#readme;,
   "_id": "async@0.9.2",
   "_shasum": "aea74d5e61c1f899613bf64bda66d4c78f2fd17d",
-  "_from": "async@~0.9.0",
+  "_from": "async@^0.9.0",
   "_npmVersion": "2.9.0",
   "_nodeVersion": "2.0.1",
   "_npmUser": {
@@ -79,5 +79,6 @@
 "tarball": "http://registry.npmjs.org/async/-/async-0.9.2.tgz;
   },
   "directories": {},
-  "_resolved": "https://registry.npmjs.org/async/-/async-0.9.2.tgz;
+  "_resolved": "https://registry.npmjs.org/async/-/async-0.9.2.tgz;,
+  "readme": "ERROR: No README data found!"
 }
diff --git 
a/node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
 
b/node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
index 6eb9115..04953c7 100644
--- 
a/node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
+++ 
b/node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
@@ -81,6 +81,5 @@
 "tarball": "http://registry.npmjs.org/core-js/-/core-js-0.9.18.tgz;
   },
   "directories": {},
-  "_resolved": "https://registry.npmjs.org/core-js/-/core-js-0.9.18.tgz;,
-  "readme": "ERROR: No README data found!"
+  "_resolved": "https://registry.npmjs.org/core-js/-/core-js-0.9.18.tgz;
 }
diff --git a/node_modules/tassembly/browser/tassembly.js 
b/node_modules/tassembly/browser/tassembly.js
index aaad289..952905c 100644
--- a/node_modules/tassembly/browser/tassembly.js
+++ b/node_modules/tassembly/browser/tassembly.js
@@ -3,15 +3,15 @@
 function(a){var b=this;return a.replace(q,function(a,c,g,m,h){return 
c?b.decodeEntity(c):g?b.decodeChar(parseInt(g,10)):m?b.decodeChar(parseInt(m,16)):h})};var
 n=RegExp("[\t 
]|\u00ad|\u1806|\u200b|\u2060|\ufeff|\u034f|\u180b|\u180c|\u180d|\u200c|\u200d|[\ufe00-\ufe0f]","g");e.prototype.cssDecodeRE=/\\(?:((?:\n|\r\n|\r|\f))|([0-9A-Fa-f]{1,6})[\x20\t\r\n\f]?|(.)|()$)/;e.prototype.sanitizeStyle=function(a){function
 b(a,b){var 
d,h;"'"===b?(d=/'/g,h=/'([^'\n\r\f]*)$/):(d=/"/g,h=/"([^"\n\r\f]*)$/);if(1===
 (a.match(d)||[]).length%2)a=a.replace(h,function(a,c){return" "+c});return 
a}a=this.decodeCharReferences(a);a=a.replace(this.cssDecodeRE,function(a,b,d,h){var
 f;if(void 0!==b)return"";if(void 
0!==d){a=parseInt(d,16);try{f=String.fromCharCode(a)}catch(e){f=a.toString()}}else
 f=void 
0!==h?h:"\\";return"\n"===f||'"'===f||"'"===f||"\\"===f?"\\"+f.charCodeAt(0).toString(16)+"
 ":f});a=a.replace(/\/\*.*\*\//g," ");a=b(a,"'");a=b(a,'"');var 
d=a.indexOf("/*");0<=d&&(a=a.substr(0,d));return/[\000-\010\016-\037\177]/.test(a)?
 "/* invalid control char 
*/":/expression|filter\s*:|accelerator\s*:|url\s*\(/i.test(a)?"/* insecure 
input */":a};e.prototype.sanitizeHref=function(a){var 
b=a.match(/^((?:[a-zA-Z][^:\/]*:)?(?:\/\/)?)([^\/]+)(\/?.*)/),d,c;if(b){if(d=b[1],c=b[2],a=b[3],!d.match(this.protocolRegex))return
 null}else c=d="";c=c.replace(n,"");return 
d+c+a};k.exports={AttributeSanitizer:e}},{}],2:[function(r,k,s){function 
e(){this.uid=0;this.cache={};this.partials={}}function l(a){var 
b="",d=-1,c="";do{if(/^$|[\[:(,]/.test(c))b+=
-c,/[pri]/.test(a[d+1])&&/^(?:p(?:[cm]s?)|r[mc]|i)(?:[\.\(\)}\[\]]|$)/.test(a.slice(d+1))&&(b+="c.");else
 
if("'"===c){if(c=a.slice(d).match(/'(?:[^\\']+|\\')*'/))b+=c[0],d+=c[0].length-1}else
 b+=c;d++;c=a[d]}while(c);return b}function 
q(a,b,d){d=d||"";a=""+a;if(n.test(a))return a=l(a),d+a;if(/^'/.test(a))return

[MediaWiki-commits] [Gerrit] Update restbase to 2da393c - change (mediawiki...deploy)

2015-11-19 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to 2da393c
..


Update restbase to 2da393c

List of changes:
xxx Update node module dependencies

Change-Id: I17e5168cc63285cdfb219479426cd23fc7a8c9ce
---
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
M node_modules/tassembly/browser/tassembly.js
M node_modules/tassembly/browser/tassembly.orig.js
M node_modules/tassembly/package.json
M node_modules/tassembly/tassembly.js
M node_modules/tassembly/test.js
7 files changed, 70 insertions(+), 35 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git 
a/node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
 
b/node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
index 76d5799..062a245 100644
--- 
a/node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
+++ 
b/node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
@@ -57,7 +57,7 @@
   "homepage": "https://github.com/caolan/async#readme;,
   "_id": "async@0.9.2",
   "_shasum": "aea74d5e61c1f899613bf64bda66d4c78f2fd17d",
-  "_from": "async@~0.9.0",
+  "_from": "async@^0.9.0",
   "_npmVersion": "2.9.0",
   "_nodeVersion": "2.0.1",
   "_npmUser": {
@@ -79,5 +79,6 @@
 "tarball": "http://registry.npmjs.org/async/-/async-0.9.2.tgz;
   },
   "directories": {},
-  "_resolved": "https://registry.npmjs.org/async/-/async-0.9.2.tgz;
+  "_resolved": "https://registry.npmjs.org/async/-/async-0.9.2.tgz;,
+  "readme": "ERROR: No README data found!"
 }
diff --git 
a/node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
 
b/node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
index 6eb9115..04953c7 100644
--- 
a/node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
+++ 
b/node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
@@ -81,6 +81,5 @@
 "tarball": "http://registry.npmjs.org/core-js/-/core-js-0.9.18.tgz;
   },
   "directories": {},
-  "_resolved": "https://registry.npmjs.org/core-js/-/core-js-0.9.18.tgz;,
-  "readme": "ERROR: No README data found!"
+  "_resolved": "https://registry.npmjs.org/core-js/-/core-js-0.9.18.tgz;
 }
diff --git a/node_modules/tassembly/browser/tassembly.js 
b/node_modules/tassembly/browser/tassembly.js
index aaad289..952905c 100644
--- a/node_modules/tassembly/browser/tassembly.js
+++ b/node_modules/tassembly/browser/tassembly.js
@@ -3,15 +3,15 @@
 function(a){var b=this;return a.replace(q,function(a,c,g,m,h){return 
c?b.decodeEntity(c):g?b.decodeChar(parseInt(g,10)):m?b.decodeChar(parseInt(m,16)):h})};var
 n=RegExp("[\t 
]|\u00ad|\u1806|\u200b|\u2060|\ufeff|\u034f|\u180b|\u180c|\u180d|\u200c|\u200d|[\ufe00-\ufe0f]","g");e.prototype.cssDecodeRE=/\\(?:((?:\n|\r\n|\r|\f))|([0-9A-Fa-f]{1,6})[\x20\t\r\n\f]?|(.)|()$)/;e.prototype.sanitizeStyle=function(a){function
 b(a,b){var 
d,h;"'"===b?(d=/'/g,h=/'([^'\n\r\f]*)$/):(d=/"/g,h=/"([^"\n\r\f]*)$/);if(1===
 (a.match(d)||[]).length%2)a=a.replace(h,function(a,c){return" "+c});return 
a}a=this.decodeCharReferences(a);a=a.replace(this.cssDecodeRE,function(a,b,d,h){var
 f;if(void 0!==b)return"";if(void 
0!==d){a=parseInt(d,16);try{f=String.fromCharCode(a)}catch(e){f=a.toString()}}else
 f=void 
0!==h?h:"\\";return"\n"===f||'"'===f||"'"===f||"\\"===f?"\\"+f.charCodeAt(0).toString(16)+"
 ":f});a=a.replace(/\/\*.*\*\//g," ");a=b(a,"'");a=b(a,'"');var 
d=a.indexOf("/*");0<=d&&(a=a.substr(0,d));return/[\000-\010\016-\037\177]/.test(a)?
 "/* invalid control char 
*/":/expression|filter\s*:|accelerator\s*:|url\s*\(/i.test(a)?"/* insecure 
input */":a};e.prototype.sanitizeHref=function(a){var 
b=a.match(/^((?:[a-zA-Z][^:\/]*:)?(?:\/\/)?)([^\/]+)(\/?.*)/),d,c;if(b){if(d=b[1],c=b[2],a=b[3],!d.match(this.protocolRegex))return
 null}else c=d="";c=c.replace(n,"");return 
d+c+a};k.exports={AttributeSanitizer:e}},{}],2:[function(r,k,s){function 
e(){this.uid=0;this.cache={};this.partials={}}function l(a){var 
b="",d=-1,c="";do{if(/^$|[\[:(,]/.test(c))b+=
-c,/[pri]/.test(a[d+1])&&/^(?:p(?:[cm]s?)|r[mc]|i)(?:[\.\(\)}\[\]]|$)/.test(a.slice(d+1))&&(b+="c.");else
 
if("'"===c){if(c=a.slice(d).match(/'(?:[^\\']+|\\')*'/))b+=c[0],d+=c[0].length-1}else
 b+=c;d++;c=a[d]}while(c);return b}function 
q(a,b,d){d=d||"";a=""+a;if(n.test(a))return a=l(a),d+a;if(/^'/.test(a))return 
d+JSON.stringify(a.slice(1,-1).replace(/\\'/g,"'"));if(/^[cm](?:\.[a-zA-Z_$]*)?$/.test(a))return
 d+a;var

[MediaWiki-commits] [Gerrit] RESTBase: Update to new specs & enable summary end point - change (operations/puppet)

2015-11-19 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/254372

Change subject: RESTBase: Update to new specs & enable summary end point
..

RESTBase: Update to new specs & enable summary end point

- RESTBase now supports passing down module parameters using its handler
  templating feature. This means that we now need to process template
  parameters against premature expansion by moving them to a new 'templates'
  object. This patch does this for a single apiRequest template.

- Enable the new page summary API end point developed with reading, which
  exposes a cacheable page summary & text extract.

Change-Id: I129bbd0a53db9584aa36ce9de6d927d9a67f0433
---
M modules/mariadb
M modules/restbase/templates/config.labs.yaml.erb
M modules/restbase/templates/config.yaml.erb
3 files changed, 51 insertions(+), 44 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/72/254372/1

diff --git a/modules/mariadb b/modules/mariadb
index 44fd087..6014a23 16
--- a/modules/mariadb
+++ b/modules/mariadb
-Subproject commit 44fd08751f4f6cfa15a6275253bf231f7533a88b
+Subproject commit 6014a235e35a8ac0ab2deadaf0de4690a4e63791
diff --git a/modules/restbase/templates/config.labs.yaml.erb 
b/modules/restbase/templates/config.labs.yaml.erb
index 66a3a4f..8d45e1d 100644
--- a/modules/restbase/templates/config.labs.yaml.erb
+++ b/modules/restbase/templates/config.labs.yaml.erb
@@ -40,20 +40,6 @@
 name: Apache2
 url: http://www.apache.org/licenses/LICENSE-2.0
 
-# Override the base path for host-based (proxied) requests. In our case,
-# we proxy https://{domain}/api/rest_v1/ to the API.
-x-host-basePath: /api/rest_v1
-
-x-modules:
-  - name: mobileapps-public
-path: specs/mediawiki/v1/mobileapps
-type: spec
-options:
-  host: <%= @mobileapps_uri %>
-x-subspecs:
-  - mediawiki/v1/content
-  - mediawiki/v1/graphoid
-  - media/v1/mathoid
 securityDefinitions: /content-security/1.0.0
   mediawiki_auth:
 description: Checks permissions using MW api
@@ -73,6 +59,30 @@
 x-whitelists:
   internal:
 - /^(?::::)?(?:10|127)\./
+
+# Override the base path for host-based (proxied) requests. In our case,
+# we proxy https://{domain}/api/rest_v1/ to the API.
+x-host-basePath: /api/rest_v1
+
+x-modules:
+  - name: mobileapps-public
+path: specs/mediawiki/v1/mobileapps.yaml
+options:
+  host: <%= @mobileapps_uri %>
+
+  - name: page-summary
+path: wikimedia/v1/summary_v1.js
+options:
+  # One hour, for now.
+  response_cache-control: 'max-age: 3600, s-maxage: 3600'
+
+  - path: wikimedia/v1/graphoid_v1.yaml
+options:
+  host: <%= @graphoid_uri %>
+
+x-subspecs:
+  - mediawiki/v1/content
+  - media/v1/mathoid
 
   wmf-sys-1.0.0: /sys/1.0.0
 info:
@@ -136,7 +146,7 @@
 x-modules:
   - name: action
 type: file
-options:
+templates:
   apiRequest:
 method: post
 uri: http://{domain}/w/api.php
@@ -155,13 +165,6 @@
 type: file
 options:
   parsoidHost: <%= @parsoid_uri %>
-
-  /{module:graphoid}/v1/png/{title}/{revision}/{graph_id}:
-get:
-  x-request-handler:
-- get_from_graphoid:
-request:
-  uri: <%= @graphoid_uri 
%>/{domain}/v1/png/{title}/{revision}/{graph_id}
 
   /{module:mobileapps}:
 x-subspec:
diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 46b1e75..872651b 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -37,20 +37,6 @@
 name: Apache2
 url: http://www.apache.org/licenses/LICENSE-2.0
 
-# Override the base path for host-based (proxied) requests. In our case,
-# we proxy https://{domain}/api/rest_v1/ to the API.
-x-host-basePath: /api/rest_v1
-
-x-modules:
-  - name: mobileapps-public
-path: specs/mediawiki/v1/mobileapps
-type: spec
-options:
-  host: <%= @mobileapps_uri %>
-x-subspecs:
-  - mediawiki/v1/content
-  - mediawiki/v1/graphoid
-  - media/v1/mathoid
 securityDefinitions: /content-security/1.0.0
   mediawiki_auth:
 description: Checks permissions using MW api
@@ -70,6 +56,31 @@
 x-whitelists:
   internal:
 - /^(?::::)?(?:10|127)\./
+
+# Override the base path for host-based (proxied) requests. In our case,
+# we proxy https://{domain}/api/rest_v1/ to the API.
+x-host-basePath: /api/rest_v1
+
+x-modules:
+  - name: mobileapps-public
+path: specs/mediawiki/v1/mobileapps.yaml
+

[MediaWiki-commits] [Gerrit] Remove HTML dump reference - change (operations/puppet)

2015-11-16 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/253468

Change subject: Remove HTML dump reference
..

Remove HTML dump reference

Change-Id: I979baaa7017efbb1564bff99be9719e4d0036b4a
---
M modules/restbase/templates/config.yaml.erb
1 file changed, 0 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/68/253468/1

diff --git a/modules/restbase/templates/config.yaml.erb 
b/modules/restbase/templates/config.yaml.erb
index 8b6e0a5..941d1c7 100644
--- a/modules/restbase/templates/config.yaml.erb
+++ b/modules/restbase/templates/config.yaml.erb
@@ -28,9 +28,6 @@
   this API.
 - Set a unique `User-Agent` header that allows us to contact you
   quickly. Email addresses or URLs of contact pages work well.
-- Consider using our [HTML
-  dumps](https://phabricator.wikimedia.org/T17017) once they
-  become available.
 
   termsOfService: https://wikimediafoundation.org/wiki/Terms_of_Use
   contact:

-- 
To view, visit https://gerrit.wikimedia.org/r/253468
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I979baaa7017efbb1564bff99be9719e4d0036b4a
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Add /api/ listing to www.wikimedia.org - change (operations/puppet)

2015-11-12 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/252863

Change subject: Add /api/ listing to www.wikimedia.org
..

Add /api/ listing to www.wikimedia.org

This is a follow-up to 1e8426d0, which added the api listing to most projects.
We missed www.wikimedia.org, which this patch rectified.

Bug: T118519
Change-Id: I99af301f51f45ba838dda2521df283c9ccf4d59e
---
M modules/mediawiki/files/apache/sites/wikimedia.conf
1 file changed, 3 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/63/252863/1

diff --git a/modules/mediawiki/files/apache/sites/wikimedia.conf 
b/modules/mediawiki/files/apache/sites/wikimedia.conf
index 713ba6c..5240943 100644
--- a/modules/mediawiki/files/apache/sites/wikimedia.conf
+++ b/modules/mediawiki/files/apache/sites/wikimedia.conf
@@ -144,6 +144,9 @@
 # Redirect wiki which was previously at this location
 RewriteRule ^/wiki/(.*)$ %{ENV:RW_PROTO}://wikimediafoundation.org/wiki/$1 
[R=301,L]
 
+# /api/ rewrites
+Include "sites-enabled/api-rewrites.incl"
+
   

 php_admin_flag engine on

-- 
To view, visit https://gerrit.wikimedia.org/r/252863
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I99af301f51f45ba838dda2521df283c9ccf4d59e
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 140518b - change (mediawiki...deploy)

2015-11-11 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/252610

Change subject: Update restbase to 140518b
..

Update restbase to 140518b

List of changes:
5c316c0 Use correct Parsoid v3 path for POST of pagebundle.
13ac27f Don't retry failed Parsoid POST requests

Change-Id: Ie410e2c88e140d120d9b369706761f2057d414e3
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/10/252610/1

diff --git a/restbase b/restbase
index 0b094b4..140518b 16
--- a/restbase
+++ b/restbase
-Subproject commit 0b094b445907e20c53885b87fffea9a0b201a59c
+Subproject commit 140518b793d95b0b3c48e6f92b8be8430a78994e

-- 
To view, visit https://gerrit.wikimedia.org/r/252610
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie410e2c88e140d120d9b369706761f2057d414e3
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 140518b - change (mediawiki...deploy)

2015-11-11 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to 140518b
..


Update restbase to 140518b

List of changes:
5c316c0 Use correct Parsoid v3 path for POST of pagebundle.
13ac27f Don't retry failed Parsoid POST requests

Change-Id: Ie410e2c88e140d120d9b369706761f2057d414e3
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/restbase b/restbase
index 0b094b4..140518b 16
--- a/restbase
+++ b/restbase
-Subproject commit 0b094b445907e20c53885b87fffea9a0b201a59c
+Subproject commit 140518b793d95b0b3c48e6f92b8be8430a78994e

-- 
To view, visit https://gerrit.wikimedia.org/r/252610
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ie410e2c88e140d120d9b369706761f2057d414e3
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 
Gerrit-Reviewer: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Support pagebundle requests with passed-in HTML/data-parsoid - change (mediawiki...parsoid)

2015-11-11 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/252575

Change subject: Support pagebundle requests with passed-in HTML/data-parsoid
..

Support pagebundle requests with passed-in HTML/data-parsoid

This end point reinstates the ability to optionally pass in pagebundle
information in HTML title/revision requests. This is used by RESTBase to speed
up revision parse requests.

Change-Id: Ic05679147a6dfc39b045f8b7d3f84c17a082e723
---
M lib/api/ParsoidService.js
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/parsoid 
refs/changes/75/252575/1

diff --git a/lib/api/ParsoidService.js b/lib/api/ParsoidService.js
index a1c962b..ae29b95 100644
--- a/lib/api/ParsoidService.js
+++ b/lib/api/ParsoidService.js
@@ -179,6 +179,7 @@
 
// v3 API routes
app.get('/:domain/v3/page/:format/:title/:revision?', v3, p, 
routes.v3Get);
+   app.post('/:domain/v3/page/pagebundle/:title/:revision?', v3, p, 
routes.v3Post);
app.post('/:domain/v3/transform/:from/to/:format/:title?/:revision?', 
v3, p, routes.v3Post);
 
// v1 API routes

-- 
To view, visit https://gerrit.wikimedia.org/r/252575
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic05679147a6dfc39b045f8b7d3f84c17a082e723
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/parsoid
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to 0b094b4 - change (mediawiki...deploy)

2015-11-10 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to 0b094b4
..


Update restbase to 0b094b4

List of changes:
0b094b4 Switch to our own copy of content-type
xxx Update node module dependencies

Change-Id: Ife88a97cb7d0ee13497dca19105a65da52499075
---
A node_modules/content-type/.travis.yml
M node_modules/content-type/index.js
M node_modules/content-type/package.json
A node_modules/content-type/test/contentType_format.js
A node_modules/content-type/test/contentType_parse.js
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/routeswitch/node_modules/async/package.json
M restbase
7 files changed, 248 insertions(+), 25 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/node_modules/content-type/.travis.yml 
b/node_modules/content-type/.travis.yml
new file mode 100644
index 000..dced2bc
--- /dev/null
+++ b/node_modules/content-type/.travis.yml
@@ -0,0 +1,12 @@
+language: node_js
+node_js:
+  - "0.6"
+  - "0.8"
+  - "0.10"
+  - "0.12"
+sudo: false
+script:
+  - "test $TRAVIS_NODE_VERSION != '0.6' || npm test"
+  - "test $TRAVIS_NODE_VERSION  = '0.6' || npm run-script test-ci"
+after_script:
+  - "test $TRAVIS_NODE_VERSION != '0.6' && npm install coveralls@2 && cat 
./coverage/lcov.info | coveralls"
diff --git a/node_modules/content-type/index.js 
b/node_modules/content-type/index.js
index 6a2ea9f..312823b 100644
--- a/node_modules/content-type/index.js
+++ b/node_modules/content-type/index.js
@@ -17,8 +17,10 @@
  * qdtext= HTAB / SP / %x21 / %x23-5B / %x5D-7E / obs-text
  * obs-text  = %x80-FF
  * quoted-pair   = "\" ( HTAB / SP / VCHAR / obs-text )
+ *
+ * Additionally, we accept "/" in parameter values.
  */
-var paramRegExp = /; *([!#$%&'\*\+\-\.\^_`\|~0-9A-Za-z]+) *= 
*("(?:[\u000b\u0020\u0021\u0023-\u005b\u005d-\u007e\u0080-\u00ff]|\\[\u000b\u0020-\u00ff])*"|[!#$%&'\*\+\-\.\^_`\|~0-9A-Za-z]+)
 */g
+var paramRegExp = /; *([!#$%&'\*\+\-\.\^_`\|~0-9A-Za-z]+) *= 
*("(?:[\u000b\u0020\u0021\u0023-\u005b\u005d-\u007e\u0080-\u00ff]|\\[\u000b\u0020-\u00ff])*"|[!#$%&'\*\+\-\.\^_`\|~\/0-9A-Za-z]+)
 */g
 var textRegExp = /^[\u000b\u0020-\u007e\u0080-\u00ff]+$/
 var tokenRegExp = /^[!#$%&'\*\+\-\.\^_`\|~0-9A-Za-z]+$/
 
diff --git a/node_modules/content-type/package.json 
b/node_modules/content-type/package.json
index 995a411..b38f994 100644
--- a/node_modules/content-type/package.json
+++ b/node_modules/content-type/package.json
@@ -16,7 +16,7 @@
   ],
   "repository": {
 "type": "git",
-"url": "https://github.com/jshttp/content-type;
+"url": "git://github.com/jshttp/content-type"
   },
   "devDependencies": {
 "istanbul": "0.3.5",
@@ -36,29 +36,13 @@
 "test-ci": "istanbul cover node_modules/mocha/bin/_mocha --report lcovonly 
-- --reporter spec --check-leaks test/",
 "test-cov": "istanbul cover node_modules/mocha/bin/_mocha -- --reporter 
dot --check-leaks test/"
   },
-  "gitHead": "3aa58f9c5a358a3634b8601602177888b4a477d8",
+  "readme": "# content-type\n\n[![NPM Version][npm-image]][npm-url]\n[![NPM 
Downloads][downloads-image]][downloads-url]\n[![Node.js 
Version][node-version-image]][node-version-url]\n[![Build 
Status][travis-image]][travis-url]\n[![Test 
Coverage][coveralls-image]][coveralls-url]\n\nCreate and parse HTTP 
Content-Type header according to RFC 7231\n\n## Installation\n\n```sh\n$ npm 
install content-type\n```\n\n## API\n\n```js\nvar contentType = 
require('content-type')\n```\n\n### contentType.parse(string)\n\n```js\nvar obj 
= contentType.parse('image/svg+xml; charset=utf-8')\n```\n\nParse a content 
type string. This will return an object with the following\nproperties 
(examples are shown for the string `'image/svg+xml; charset=utf-8'`):\n\n - 
`type`: The media type (the type and subtype, always lower case).\n   Example: 
`'image/svg+xml'`\n\n - `parameters`: An object of the parameters in the media 
type (name of parameter\n   always lower case). Example: `{charset: 
'utf-8'}`\n\nThrows a `TypeError` if the string is missing or invalid.\n\n### 
contentType.parse(req)\n\n```js\nvar obj = contentType.parse(req)\n```\n\nParse 
the `content-type` header from the given `req`. Short-cut 
for\n`contentType.parse(req.headers['content-type'])`.\n\nThrows a `TypeError` 
if the `Content-Type` header is missing or invalid.\n\n### 
contentType.parse(res)\n\n```js\nvar obj = contentType.parse(res)\n```\n\nParse 
the `content-type` header set on the given `res`. Short-cut 
for\n`contentType.parse(res.getHeader('content-type'))`.\n\nThrows a 
`TypeError` if the `Content-Type` header is missing or invalid.\n\n### 
contentType.format(obj)\n\n```js\nvar str = contentType.format({type: 
'image/svg+xml'})\n```\n\nFormat an object into a content type string. This 
will return a string of the\ncontent type for the given object with the 
following properties (examples are\nshown that

[MediaWiki-commits] [Gerrit] Update restbase to 0b094b4 - change (mediawiki...deploy)

2015-11-10 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/252384

Change subject: Update restbase to 0b094b4
..

Update restbase to 0b094b4

List of changes:
0b094b4 Switch to our own copy of content-type
xxx Update node module dependencies

Change-Id: Ife88a97cb7d0ee13497dca19105a65da52499075
---
A node_modules/content-type/.travis.yml
M node_modules/content-type/index.js
M node_modules/content-type/package.json
A node_modules/content-type/test/contentType_format.js
A node_modules/content-type/test/contentType_parse.js
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/routeswitch/node_modules/async/package.json
M restbase
7 files changed, 248 insertions(+), 25 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/84/252384/1

diff --git a/node_modules/content-type/.travis.yml 
b/node_modules/content-type/.travis.yml
new file mode 100644
index 000..dced2bc
--- /dev/null
+++ b/node_modules/content-type/.travis.yml
@@ -0,0 +1,12 @@
+language: node_js
+node_js:
+  - "0.6"
+  - "0.8"
+  - "0.10"
+  - "0.12"
+sudo: false
+script:
+  - "test $TRAVIS_NODE_VERSION != '0.6' || npm test"
+  - "test $TRAVIS_NODE_VERSION  = '0.6' || npm run-script test-ci"
+after_script:
+  - "test $TRAVIS_NODE_VERSION != '0.6' && npm install coveralls@2 && cat 
./coverage/lcov.info | coveralls"
diff --git a/node_modules/content-type/index.js 
b/node_modules/content-type/index.js
index 6a2ea9f..312823b 100644
--- a/node_modules/content-type/index.js
+++ b/node_modules/content-type/index.js
@@ -17,8 +17,10 @@
  * qdtext= HTAB / SP / %x21 / %x23-5B / %x5D-7E / obs-text
  * obs-text  = %x80-FF
  * quoted-pair   = "\" ( HTAB / SP / VCHAR / obs-text )
+ *
+ * Additionally, we accept "/" in parameter values.
  */
-var paramRegExp = /; *([!#$%&'\*\+\-\.\^_`\|~0-9A-Za-z]+) *= 
*("(?:[\u000b\u0020\u0021\u0023-\u005b\u005d-\u007e\u0080-\u00ff]|\\[\u000b\u0020-\u00ff])*"|[!#$%&'\*\+\-\.\^_`\|~0-9A-Za-z]+)
 */g
+var paramRegExp = /; *([!#$%&'\*\+\-\.\^_`\|~0-9A-Za-z]+) *= 
*("(?:[\u000b\u0020\u0021\u0023-\u005b\u005d-\u007e\u0080-\u00ff]|\\[\u000b\u0020-\u00ff])*"|[!#$%&'\*\+\-\.\^_`\|~\/0-9A-Za-z]+)
 */g
 var textRegExp = /^[\u000b\u0020-\u007e\u0080-\u00ff]+$/
 var tokenRegExp = /^[!#$%&'\*\+\-\.\^_`\|~0-9A-Za-z]+$/
 
diff --git a/node_modules/content-type/package.json 
b/node_modules/content-type/package.json
index 995a411..b38f994 100644
--- a/node_modules/content-type/package.json
+++ b/node_modules/content-type/package.json
@@ -16,7 +16,7 @@
   ],
   "repository": {
 "type": "git",
-"url": "https://github.com/jshttp/content-type;
+"url": "git://github.com/jshttp/content-type"
   },
   "devDependencies": {
 "istanbul": "0.3.5",
@@ -36,29 +36,13 @@
 "test-ci": "istanbul cover node_modules/mocha/bin/_mocha --report lcovonly 
-- --reporter spec --check-leaks test/",
 "test-cov": "istanbul cover node_modules/mocha/bin/_mocha -- --reporter 
dot --check-leaks test/"
   },
-  "gitHead": "3aa58f9c5a358a3634b8601602177888b4a477d8",
+  "readme": "# content-type\n\n[![NPM Version][npm-image]][npm-url]\n[![NPM 
Downloads][downloads-image]][downloads-url]\n[![Node.js 
Version][node-version-image]][node-version-url]\n[![Build 
Status][travis-image]][travis-url]\n[![Test 
Coverage][coveralls-image]][coveralls-url]\n\nCreate and parse HTTP 
Content-Type header according to RFC 7231\n\n## Installation\n\n```sh\n$ npm 
install content-type\n```\n\n## API\n\n```js\nvar contentType = 
require('content-type')\n```\n\n### contentType.parse(string)\n\n```js\nvar obj 
= contentType.parse('image/svg+xml; charset=utf-8')\n```\n\nParse a content 
type string. This will return an object with the following\nproperties 
(examples are shown for the string `'image/svg+xml; charset=utf-8'`):\n\n - 
`type`: The media type (the type and subtype, always lower case).\n   Example: 
`'image/svg+xml'`\n\n - `parameters`: An object of the parameters in the media 
type (name of parameter\n   always lower case). Example: `{charset: 
'utf-8'}`\n\nThrows a `TypeError` if the string is missing or invalid.\n\n### 
contentType.parse(req)\n\n```js\nvar obj = contentType.parse(req)\n```\n\nParse 
the `content-type` header from the given `req`. Short-cut 
for\n`contentType.parse(req.headers['content-type'])`.\n\nThrows a `TypeError` 
if the `Content-Type` header is missing or invalid.\n\n### 
contentType.parse(res)\n\n```js\nvar obj = contentType.parse(res)\n```\n\nParse 
the `content-type` header set on the given `res`. Short-cut 
for\n`contentType.parse(res.getHeader('content-type'))`.\n\nThrows a 
`TypeError` if the `Content-Type` header is missing or invalid.\n\n### 
contentType.format(obj)\n\n```js\nvar str = contentType.format({type: 
'image/svg+xml'})\n```\n\nFormat an object into a content type string. This 
will return a string of the\ncontent type for

[MediaWiki-commits] [Gerrit] Update restbase to b271cb9 - change (mediawiki...deploy)

2015-10-28 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/249448

Change subject: Update restbase to b271cb9
..

Update restbase to b271cb9

List of changes:
a47af70 Save space by flattening the per article table
51c4ce7 Added a simple request validator
64b578d Shorten column names to save more space
e1f6388 Added a comment about 'required' array and a test
0667b57 Added a x-host-basePath to the example wikimedia config
6d66f2f Added a .dockerignore
668db1d Update dependencies and use a valid SPDX licence code
6e8b517 Switch to the new per-article-flat handler
5a00677 Do not DOS ourselves by re-rendering the same title in monitoring
348227d Stop testing with iojs
abc07e7 Back-end services: Strip content-length and content-encoding headers
xxx Update node module dependencies

Change-Id: I2c88ae104355fb9c6aff598002a429d4130bf91c
---
A node_modules/ajv/LICENSE
A node_modules/ajv/lib/ajv.js
A node_modules/ajv/lib/cache.js
A node_modules/ajv/lib/compile/_rules.js
A node_modules/ajv/lib/compile/equal.js
A node_modules/ajv/lib/compile/formats.js
A node_modules/ajv/lib/compile/index.js
A node_modules/ajv/lib/compile/resolve.js
A node_modules/ajv/lib/compile/rules.js
A node_modules/ajv/lib/compile/schema_obj.js
A node_modules/ajv/lib/compile/util.js
A node_modules/ajv/lib/dot/allOf.jst
A node_modules/ajv/lib/dot/anyOf.jst
A node_modules/ajv/lib/dot/definitions.def
A node_modules/ajv/lib/dot/dependencies.jst
A node_modules/ajv/lib/dot/enum.jst
A node_modules/ajv/lib/dot/format.jst
A node_modules/ajv/lib/dot/items.jst
A node_modules/ajv/lib/dot/maxItems.jst
A node_modules/ajv/lib/dot/maxLength.jst
A node_modules/ajv/lib/dot/maxProperties.jst
A node_modules/ajv/lib/dot/maximum.jst
A node_modules/ajv/lib/dot/minItems.jst
A node_modules/ajv/lib/dot/minLength.jst
A node_modules/ajv/lib/dot/minProperties.jst
A node_modules/ajv/lib/dot/minimum.jst
A node_modules/ajv/lib/dot/multipleOf.jst
A node_modules/ajv/lib/dot/not.jst
A node_modules/ajv/lib/dot/oneOf.jst
A node_modules/ajv/lib/dot/pattern.jst
A node_modules/ajv/lib/dot/properties.jst
A node_modules/ajv/lib/dot/ref.jst
A node_modules/ajv/lib/dot/required.jst
A node_modules/ajv/lib/dot/uniqueItems.jst
A node_modules/ajv/lib/dot/validate.jst
A node_modules/ajv/lib/dotjs/allOf.js
A node_modules/ajv/lib/dotjs/anyOf.js
A node_modules/ajv/lib/dotjs/dependencies.js
A node_modules/ajv/lib/dotjs/enum.js
A node_modules/ajv/lib/dotjs/format.js
A node_modules/ajv/lib/dotjs/items.js
A node_modules/ajv/lib/dotjs/maxItems.js
A node_modules/ajv/lib/dotjs/maxLength.js
A node_modules/ajv/lib/dotjs/maxProperties.js
A node_modules/ajv/lib/dotjs/maximum.js
A node_modules/ajv/lib/dotjs/minItems.js
A node_modules/ajv/lib/dotjs/minLength.js
A node_modules/ajv/lib/dotjs/minProperties.js
A node_modules/ajv/lib/dotjs/minimum.js
A node_modules/ajv/lib/dotjs/multipleOf.js
A node_modules/ajv/lib/dotjs/not.js
A node_modules/ajv/lib/dotjs/oneOf.js
A node_modules/ajv/lib/dotjs/pattern.js
A node_modules/ajv/lib/dotjs/properties.js
A node_modules/ajv/lib/dotjs/ref.js
A node_modules/ajv/lib/dotjs/required.js
A node_modules/ajv/lib/dotjs/uniqueItems.js
A node_modules/ajv/lib/dotjs/validate.js
A node_modules/ajv/lib/refs/json-schema-draft-04.json
A node_modules/ajv/package.json
R 
node_modules/busboy/node_modules/readable-stream/node_modules/core-util-is/float.patch
R 
node_modules/busboy/node_modules/readable-stream/node_modules/core-util-is/lib/util.js
C 
node_modules/busboy/node_modules/readable-stream/node_modules/core-util-is/package.json
R 
node_modules/busboy/node_modules/readable-stream/node_modules/core-util-is/util.js
R node_modules/busboy/node_modules/readable-stream/node_modules/inherits/LICENSE
R 
node_modules/busboy/node_modules/readable-stream/node_modules/inherits/inherits.js
R 
node_modules/busboy/node_modules/readable-stream/node_modules/inherits/inherits_browser.js
C 
node_modules/busboy/node_modules/readable-stream/node_modules/inherits/package.json
R node_modules/busboy/node_modules/readable-stream/node_modules/inherits/test.js
R 
node_modules/busboy/node_modules/readable-stream/node_modules/isarray/build/build.js
R 
node_modules/busboy/node_modules/readable-stream/node_modules/isarray/component.json
R node_modules/busboy/node_modules/readable-stream/node_modules/isarray/index.js
A 
node_modules/busboy/node_modules/readable-stream/node_modules/isarray/package.json
R 
node_modules/busboy/node_modules/readable-stream/node_modules/string_decoder/.npmignore
R 
node_modules/busboy/node_modules/readable-stream/node_modules/string_decoder/LICENSE
R 
node_modules/busboy/node_modules/readable-stream/node_modules/string_decoder/index.js
C 
node_modules/busboy/node_modules/readable-stream/node_modules/string_decoder/package.json
R node_modules/cassandra-uuid/node_modules/long/.npmignore
R node_modules/cassandra-uuid/node_modules/long/.travis.yml
R node_modules/cassandra-uuid/node_modules/long/LICENSE
R

[MediaWiki-commits] [Gerrit] Update restbase to b271cb9 - change (mediawiki...deploy)

2015-10-28 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/249455

Change subject: Update restbase to b271cb9
..

Update restbase to b271cb9

List of changes:
a47af70 Save space by flattening the per article table
51c4ce7 Added a simple request validator
64b578d Shorten column names to save more space
e1f6388 Added a comment about 'required' array and a test
0667b57 Added a x-host-basePath to the example wikimedia config
6d66f2f Added a .dockerignore
668db1d Update dependencies and use a valid SPDX licence code
6e8b517 Switch to the new per-article-flat handler
5a00677 Do not DOS ourselves by re-rendering the same title in monitoring
348227d Stop testing with iojs
abc07e7 Back-end services: Strip content-length and content-encoding headers
xxx Update node module dependencies

Change-Id: I271ce9fbca084efb1b6104fdc89737487364782c
---
A node_modules/ajv/LICENSE
A node_modules/ajv/lib/ajv.js
A node_modules/ajv/lib/cache.js
A node_modules/ajv/lib/compile/_rules.js
A node_modules/ajv/lib/compile/equal.js
A node_modules/ajv/lib/compile/formats.js
A node_modules/ajv/lib/compile/index.js
A node_modules/ajv/lib/compile/resolve.js
A node_modules/ajv/lib/compile/rules.js
A node_modules/ajv/lib/compile/schema_obj.js
A node_modules/ajv/lib/compile/util.js
A node_modules/ajv/lib/dot/allOf.jst
A node_modules/ajv/lib/dot/anyOf.jst
A node_modules/ajv/lib/dot/definitions.def
A node_modules/ajv/lib/dot/dependencies.jst
A node_modules/ajv/lib/dot/enum.jst
A node_modules/ajv/lib/dot/format.jst
A node_modules/ajv/lib/dot/items.jst
A node_modules/ajv/lib/dot/maxItems.jst
A node_modules/ajv/lib/dot/maxLength.jst
A node_modules/ajv/lib/dot/maxProperties.jst
A node_modules/ajv/lib/dot/maximum.jst
A node_modules/ajv/lib/dot/minItems.jst
A node_modules/ajv/lib/dot/minLength.jst
A node_modules/ajv/lib/dot/minProperties.jst
A node_modules/ajv/lib/dot/minimum.jst
A node_modules/ajv/lib/dot/multipleOf.jst
A node_modules/ajv/lib/dot/not.jst
A node_modules/ajv/lib/dot/oneOf.jst
A node_modules/ajv/lib/dot/pattern.jst
A node_modules/ajv/lib/dot/properties.jst
A node_modules/ajv/lib/dot/ref.jst
A node_modules/ajv/lib/dot/required.jst
A node_modules/ajv/lib/dot/uniqueItems.jst
A node_modules/ajv/lib/dot/validate.jst
A node_modules/ajv/lib/dotjs/allOf.js
A node_modules/ajv/lib/dotjs/anyOf.js
A node_modules/ajv/lib/dotjs/dependencies.js
A node_modules/ajv/lib/dotjs/enum.js
A node_modules/ajv/lib/dotjs/format.js
A node_modules/ajv/lib/dotjs/items.js
A node_modules/ajv/lib/dotjs/maxItems.js
A node_modules/ajv/lib/dotjs/maxLength.js
A node_modules/ajv/lib/dotjs/maxProperties.js
A node_modules/ajv/lib/dotjs/maximum.js
A node_modules/ajv/lib/dotjs/minItems.js
A node_modules/ajv/lib/dotjs/minLength.js
A node_modules/ajv/lib/dotjs/minProperties.js
A node_modules/ajv/lib/dotjs/minimum.js
A node_modules/ajv/lib/dotjs/multipleOf.js
A node_modules/ajv/lib/dotjs/not.js
A node_modules/ajv/lib/dotjs/oneOf.js
A node_modules/ajv/lib/dotjs/pattern.js
A node_modules/ajv/lib/dotjs/properties.js
A node_modules/ajv/lib/dotjs/ref.js
A node_modules/ajv/lib/dotjs/required.js
A node_modules/ajv/lib/dotjs/uniqueItems.js
A node_modules/ajv/lib/dotjs/validate.js
A node_modules/ajv/lib/refs/json-schema-draft-04.json
A node_modules/ajv/package.json
R 
node_modules/busboy/node_modules/readable-stream/node_modules/core-util-is/float.patch
R 
node_modules/busboy/node_modules/readable-stream/node_modules/core-util-is/lib/util.js
R 
node_modules/busboy/node_modules/readable-stream/node_modules/core-util-is/package.json
R 
node_modules/busboy/node_modules/readable-stream/node_modules/core-util-is/util.js
R node_modules/busboy/node_modules/readable-stream/node_modules/inherits/LICENSE
R 
node_modules/busboy/node_modules/readable-stream/node_modules/inherits/inherits.js
R 
node_modules/busboy/node_modules/readable-stream/node_modules/inherits/inherits_browser.js
C 
node_modules/busboy/node_modules/readable-stream/node_modules/inherits/package.json
R node_modules/busboy/node_modules/readable-stream/node_modules/inherits/test.js
R 
node_modules/busboy/node_modules/readable-stream/node_modules/isarray/build/build.js
R 
node_modules/busboy/node_modules/readable-stream/node_modules/isarray/component.json
R node_modules/busboy/node_modules/readable-stream/node_modules/isarray/index.js
A 
node_modules/busboy/node_modules/readable-stream/node_modules/isarray/package.json
R 
node_modules/busboy/node_modules/readable-stream/node_modules/string_decoder/.npmignore
R 
node_modules/busboy/node_modules/readable-stream/node_modules/string_decoder/LICENSE
R 
node_modules/busboy/node_modules/readable-stream/node_modules/string_decoder/index.js
C 
node_modules/busboy/node_modules/readable-stream/node_modules/string_decoder/package.json
R node_modules/cassandra-uuid/node_modules/long/.npmignore
R node_modules/cassandra-uuid/node_modules/long/.travis.yml
R node_modules/cassandra-uuid/node_modules/long/LICENSE
R

[MediaWiki-commits] [Gerrit] Update restbase to b271cb9 - change (mediawiki...deploy)

2015-10-28 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/249458

Change subject: Update restbase to b271cb9
..

Update restbase to b271cb9

List of changes:
a47af70 Save space by flattening the per article table
51c4ce7 Added a simple request validator
64b578d Shorten column names to save more space
e1f6388 Added a comment about 'required' array and a test
0667b57 Added a x-host-basePath to the example wikimedia config
6d66f2f Added a .dockerignore
668db1d Update dependencies and use a valid SPDX licence code
6e8b517 Switch to the new per-article-flat handler
5a00677 Do not DOS ourselves by re-rendering the same title in monitoring
348227d Stop testing with iojs
abc07e7 Back-end services: Strip content-length and content-encoding headers
xxx Update node module dependencies

Change-Id: I421ba80bb687688b69237a711dd7bb7e3946cd5c
---
A node_modules/ajv/LICENSE
A node_modules/ajv/lib/ajv.js
A node_modules/ajv/lib/cache.js
A node_modules/ajv/lib/compile/_rules.js
A node_modules/ajv/lib/compile/equal.js
A node_modules/ajv/lib/compile/formats.js
A node_modules/ajv/lib/compile/index.js
A node_modules/ajv/lib/compile/resolve.js
A node_modules/ajv/lib/compile/rules.js
A node_modules/ajv/lib/compile/schema_obj.js
A node_modules/ajv/lib/compile/util.js
A node_modules/ajv/lib/dot/allOf.jst
A node_modules/ajv/lib/dot/anyOf.jst
A node_modules/ajv/lib/dot/definitions.def
A node_modules/ajv/lib/dot/dependencies.jst
A node_modules/ajv/lib/dot/enum.jst
A node_modules/ajv/lib/dot/format.jst
A node_modules/ajv/lib/dot/items.jst
A node_modules/ajv/lib/dot/maxItems.jst
A node_modules/ajv/lib/dot/maxLength.jst
A node_modules/ajv/lib/dot/maxProperties.jst
A node_modules/ajv/lib/dot/maximum.jst
A node_modules/ajv/lib/dot/minItems.jst
A node_modules/ajv/lib/dot/minLength.jst
A node_modules/ajv/lib/dot/minProperties.jst
A node_modules/ajv/lib/dot/minimum.jst
A node_modules/ajv/lib/dot/multipleOf.jst
A node_modules/ajv/lib/dot/not.jst
A node_modules/ajv/lib/dot/oneOf.jst
A node_modules/ajv/lib/dot/pattern.jst
A node_modules/ajv/lib/dot/properties.jst
A node_modules/ajv/lib/dot/ref.jst
A node_modules/ajv/lib/dot/required.jst
A node_modules/ajv/lib/dot/uniqueItems.jst
A node_modules/ajv/lib/dot/validate.jst
A node_modules/ajv/lib/dotjs/allOf.js
A node_modules/ajv/lib/dotjs/anyOf.js
A node_modules/ajv/lib/dotjs/dependencies.js
A node_modules/ajv/lib/dotjs/enum.js
A node_modules/ajv/lib/dotjs/format.js
A node_modules/ajv/lib/dotjs/items.js
A node_modules/ajv/lib/dotjs/maxItems.js
A node_modules/ajv/lib/dotjs/maxLength.js
A node_modules/ajv/lib/dotjs/maxProperties.js
A node_modules/ajv/lib/dotjs/maximum.js
A node_modules/ajv/lib/dotjs/minItems.js
A node_modules/ajv/lib/dotjs/minLength.js
A node_modules/ajv/lib/dotjs/minProperties.js
A node_modules/ajv/lib/dotjs/minimum.js
A node_modules/ajv/lib/dotjs/multipleOf.js
A node_modules/ajv/lib/dotjs/not.js
A node_modules/ajv/lib/dotjs/oneOf.js
A node_modules/ajv/lib/dotjs/pattern.js
A node_modules/ajv/lib/dotjs/properties.js
A node_modules/ajv/lib/dotjs/ref.js
A node_modules/ajv/lib/dotjs/required.js
A node_modules/ajv/lib/dotjs/uniqueItems.js
A node_modules/ajv/lib/dotjs/validate.js
A node_modules/ajv/lib/refs/json-schema-draft-04.json
A node_modules/ajv/package.json
M node_modules/core-js/bower.json
M node_modules/core-js/client/core.js
M node_modules/core-js/client/core.min.js
M node_modules/core-js/client/core.min.js.map
M node_modules/core-js/client/library.js
M node_modules/core-js/client/library.min.js
M node_modules/core-js/client/library.min.js.map
M node_modules/core-js/client/shim.js
M node_modules/core-js/client/shim.min.js
M node_modules/core-js/client/shim.min.js.map
M node_modules/core-js/library/modules/$.core.js
M node_modules/core-js/library/modules/$.iobject.js
M node_modules/core-js/library/modules/$.iter-define.js
A node_modules/core-js/library/modules/$.species-constructor.js
M node_modules/core-js/library/modules/$.typed-array.js
M node_modules/core-js/library/modules/es6.math.imul.js
M node_modules/core-js/library/modules/es6.promise.js
M node_modules/core-js/modules/$.core.js
M node_modules/core-js/modules/$.iobject.js
M node_modules/core-js/modules/$.iter-define.js
M node_modules/core-js/modules/$.redef.js
A node_modules/core-js/modules/$.species-constructor.js
M node_modules/core-js/modules/$.typed-array.js
M node_modules/core-js/modules/es6.math.imul.js
M node_modules/core-js/modules/es6.promise.js
M node_modules/core-js/package.json
M node_modules/htcp-purge/package.json
M node_modules/js-yaml/node_modules/argparse/lib/argument_parser.js
M node_modules/js-yaml/node_modules/argparse/package.json
M node_modules/js-yaml/node_modules/esprima/ChangeLog
M node_modules/js-yaml/node_modules/esprima/esprima.js
M node_modules/js-yaml/node_modules/esprima/package.json
M node_modules/js-yaml/package.json
M node_modules/jsonwebtoken/index.js
M node_modules/jsonwebtoken/package.json
A

[MediaWiki-commits] [Gerrit] Update restbase to 176eebd - change (mediawiki...deploy)

2015-10-02 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to 176eebd
..


Update restbase to 176eebd

List of changes:
xxx Update node module dependencies

Change-Id: I8972373cb7164b8d373ddae8bc5e4f9250bc34df
---
M node_modules/js-yaml/node_modules/argparse/package.json
M 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/inherits/package.json
M 
node_modules/jsonwebtoken/node_modules/jws/node_modules/jwa/node_modules/ecdsa-sig-formatter/node_modules/asn1.js/node_modules/inherits/package.json
M 
node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/inherits/package.json
M 
node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/isarray/package.json
M 
node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/string_decoder/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/core-js/package.json
A node_modules/service-runner/.jscs.json
D node_modules/service-runner/.jscsrc
A node_modules/service-runner/.jshintignore
A node_modules/service-runner/.travis.yml
M node_modules/service-runner/lib/base_service.js
M node_modules/service-runner/lib/docker.js
M node_modules/service-runner/lib/logger.js
M node_modules/service-runner/lib/master.js
M node_modules/service-runner/lib/worker.js
M 
node_modules/service-runner/node_modules/yargs/node_modules/cliui/node_modules/right-align/node_modules/align-text/package.json
M node_modules/service-runner/package.json
A node_modules/service-runner/test.js
A node_modules/service-runner/test/index.js
M 
node_modules/swagger-router/node_modules/js-yaml/node_modules/argparse/package.json
21 files changed, 509 insertions(+), 137 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/node_modules/js-yaml/node_modules/argparse/package.json 
b/node_modules/js-yaml/node_modules/argparse/package.json
index 4ff4f69..959b779 100644
--- a/node_modules/js-yaml/node_modules/argparse/package.json
+++ b/node_modules/js-yaml/node_modules/argparse/package.json
@@ -40,7 +40,7 @@
   "gitHead": "990f1b5332e70dd3c1c437d2f4077a2b63ac9674",
   "_id": "argparse@1.0.2",
   "_shasum": "bcfae39059656d1973d0b9e6a1a74154b5a9a136",
-  "_from": "argparse@~1.0.2",
+  "_from": "argparse@~ 1.0.0",
   "_npmVersion": "1.4.28",
   "_npmUser": {
 "name": "vitaly",
diff --git 
a/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/inherits/package.json
 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/inherits/package.json
index 292dc68..420a206 100644
--- 
a/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/inherits/package.json
+++ 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/inherits/package.json
@@ -30,7 +30,7 @@
 "shasum": "b17d08d326b4423e568eff719f91b0b1cbdf69f1",
 "tarball": "http://registry.npmjs.org/inherits/-/inherits-2.0.1.tgz;
   },
-  "_from": "inherits@~2.0.1",
+  "_from": "inherits@^2.0.1",
   "_npmVersion": "1.3.8",
   "_npmUser": {
 "name": "isaacs",
diff --git 
a/node_modules/jsonwebtoken/node_modules/jws/node_modules/jwa/node_modules/ecdsa-sig-formatter/node_modules/asn1.js/node_modules/inherits/package.json
 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/jwa/node_modules/ecdsa-sig-formatter/node_modules/asn1.js/node_modules/inherits/package.json
index 420a206..2ae619b 100644
--- 
a/node_modules/jsonwebtoken/node_modules/jws/node_modules/jwa/node_modules/ecdsa-sig-formatter/node_modules/asn1.js/node_modules/inherits/package.json
+++ 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/jwa/node_modules/ecdsa-sig-formatter/node_modules/asn1.js/node_modules/inherits/package.json
@@ -30,7 +30,7 @@
 "shasum": "b17d08d326b4423e568eff719f91b0b1cbdf69f1",
 "tarball": "http://registry.npmjs.org/inherits/-/inherits-2.0.1.tgz;
   },
-  "_from": "inherits@^2.0.1",
+  "_from": "inherits@2",
   "_npmVersion": "1.3.8",
   "_npmUser": {
 "name": "isaacs",
diff --git 
a/node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/inherits/package.json
 
b/node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/inherits/package.json
index 420a206..75ae68e 100644
--- 
a/node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/inherits/package.json
+++ 
b/node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/inherits/package.json
@@ -30,7 +30,7 @@
 "shasum": "b17d08d326b4423e568eff719f91b0b1cbdf69f1",
 "tarball":

[MediaWiki-commits] [Gerrit] Update restbase to 540d15d - change (mediawiki...deploy)

2015-09-18 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239436

Change subject: Update restbase to 540d15d
..

Update restbase to 540d15d

List of changes:
f5c8145 Added support for default values in templates
445db98 Use expression parser for complex templates
ce6cb1c Provide an errorHandler at compile time too
3243496 Grouped mobileapps sys endpoints together
0cc75f9 Removed v1 from sys mobileapps endpoints
ebaaef1 Removed the delay from resource handler and check resources length
xxx Update node module dependencies

Change-Id: I2623449c3fb7f985d90388e1d77a7097d82b6605
---
M node_modules/js-yaml/node_modules/argparse/package.json
M 
node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/core-util-is/package.json
M 
node_modules/jsonwebtoken/node_modules/jws/node_modules/jwa/node_modules/ecdsa-sig-formatter/node_modules/asn1.js/node_modules/inherits/package.json
M 
node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/core-util-is/package.json
M 
node_modules/preq/node_modules/request/node_modules/bl/node_modules/readable-stream/node_modules/inherits/package.json
M node_modules/restbase-mod-table-cassandra/lib/db.js
M node_modules/restbase-mod-table-cassandra/lib/dbutils.js
M node_modules/restbase-mod-table-cassandra/lib/schemaMigration.js
M 
node_modules/restbase-mod-table-cassandra/node_modules/cassandra-driver/node_modules/async/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/node_modules/routeswitch/node_modules/async/package.json
M 
node_modules/restbase-mod-table-cassandra/node_modules/restbase-mod-table-spec/package.json
M node_modules/restbase-mod-table-cassandra/package.json
D node_modules/restbase-mod-table-cassandra/tmp/create-varint-tbl.json
D node_modules/restbase-mod-table-cassandra/tmp/get-varint-range.json
D node_modules/restbase-mod-table-cassandra/tmp/insert-varint.json
D node_modules/restbase-mod-table-cassandra/tmp/proba.js
M 
node_modules/service-runner/node_modules/bunyan/node_modules/mv/node_modules/rimraf/node_modules/glob/node_modules/inherits/package.json
M 
node_modules/service-runner/node_modules/yargs/node_modules/cliui/node_modules/right-align/node_modules/align-text/node_modules/kind-of/package.json
M 
node_modules/service-runner/node_modules/yargs/node_modules/cliui/node_modules/right-align/node_modules/align-text/node_modules/repeat-string/package.json
M 
node_modules/service-runner/node_modules/yargs/node_modules/cliui/node_modules/right-align/node_modules/align-text/package.json
M 
node_modules/swagger-router/node_modules/js-yaml/node_modules/argparse/package.json
A node_modules/template-expression-compiler/.npmignore
A node_modules/template-expression-compiler/.travis.yml
A node_modules/template-expression-compiler/ExpressionParser.js
A node_modules/template-expression-compiler/ExpressionParser.peg.js
A node_modules/template-expression-compiler/LICENSE
A node_modules/template-expression-compiler/buildExpressionParser.js
A node_modules/template-expression-compiler/index.js
A node_modules/template-expression-compiler/package.json
A node_modules/template-expression-compiler/test/index.js
M restbase
31 files changed, 2,080 insertions(+), 170 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/36/239436/1

diff --git a/node_modules/js-yaml/node_modules/argparse/package.json 
b/node_modules/js-yaml/node_modules/argparse/package.json
index 959b779..4ff4f69 100644
--- a/node_modules/js-yaml/node_modules/argparse/package.json
+++ b/node_modules/js-yaml/node_modules/argparse/package.json
@@ -40,7 +40,7 @@
   "gitHead": "990f1b5332e70dd3c1c437d2f4077a2b63ac9674",
   "_id": "argparse@1.0.2",
   "_shasum": "bcfae39059656d1973d0b9e6a1a74154b5a9a136",
-  "_from": "argparse@~ 1.0.0",
+  "_from": "argparse@~1.0.2",
   "_npmVersion": "1.4.28",
   "_npmUser": {
 "name": "vitaly",
diff --git 
a/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/core-util-is/package.json
 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/core-util-is/package.json
index 2b7593c..add87ed 100644
--- 
a/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/core-util-is/package.json
+++ 
b/node_modules/jsonwebtoken/node_modules/jws/node_modules/base64url/node_modules/concat-stream/node_modules/readable-stream/node_modules/core-util-is/package.json
@@ -49,6 +49,5 @@
   ],
   "directories": {},
   "_shasum": "6b07085aef9a3ccac6ee53bf9d3df0c1521a5538",
-  "_resolved": 
"https://registry.npmjs.org/core-util-is/-/core-util-is-1.0.1.tgz;,
-

[MediaWiki-commits] [Gerrit] Update restbase to daacf4d - change (mediawiki...deploy)

2015-09-18 Thread GWicke (Code Review)

GWicke has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239440

Change subject: Update restbase to daacf4d
..

Update restbase to daacf4d

List of changes:
a2ab2ad Revert "Removed v1 from sys mobileapps endpoints"

Change-Id: Ic1d449db28ec5a561f8669eae154217e41b3bb63
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/restbase/deploy 
refs/changes/40/239440/1

diff --git a/restbase b/restbase
index 540d15d..daacf4d 16
--- a/restbase
+++ b/restbase
-Subproject commit 540d15da7dcb09afea12d50f25b116f69936745b
+Subproject commit daacf4daa109bb53ebe2cf1b5d8ab02ac63f3ed0

-- 
To view, visit https://gerrit.wikimedia.org/r/239440
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic1d449db28ec5a561f8669eae154217e41b3bb63
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Update restbase to daacf4d - change (mediawiki...deploy)

2015-09-18 Thread GWicke (Code Review)

GWicke has submitted this change and it was merged.

Change subject: Update restbase to daacf4d
..


Update restbase to daacf4d

List of changes:
a2ab2ad Revert "Removed v1 from sys mobileapps endpoints"

Change-Id: Ic1d449db28ec5a561f8669eae154217e41b3bb63
---
M restbase
1 file changed, 0 insertions(+), 0 deletions(-)

Approvals:
  GWicke: Verified; Looks good to me, approved



diff --git a/restbase b/restbase
index 540d15d..daacf4d 16
--- a/restbase
+++ b/restbase
-Subproject commit 540d15da7dcb09afea12d50f25b116f69936745b
+Subproject commit daacf4daa109bb53ebe2cf1b5d8ab02ac63f3ed0

-- 
To view, visit https://gerrit.wikimedia.org/r/239440
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ic1d449db28ec5a561f8669eae154217e41b3bb63
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/restbase/deploy
Gerrit-Branch: master
Gerrit-Owner: GWicke 
Gerrit-Reviewer: GWicke 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

1 2 3 4 5 6 7 8 9 10 >

1 - 100 of 2235 matches

Mail list logo