from:"Awight \(Code Review\)"

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Minor fixes to the wikilabels role

2018-01-18 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/405035 )

Change subject: Minor fixes to the wikilabels role
..

Minor fixes to the wikilabels role

Change-Id: I1182e76d094eca9a2ea3ede425d32e349b3e4132
---
M puppet/modules/wikilabels/manifests/init.pp
1 file changed, 4 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/35/405035/1

diff --git a/puppet/modules/wikilabels/manifests/init.pp 
b/puppet/modules/wikilabels/manifests/init.pp
index fb8ee4c..1e13f22 100644
--- a/puppet/modules/wikilabels/manifests/init.pp
+++ b/puppet/modules/wikilabels/manifests/init.pp
@@ -8,13 +8,13 @@
 #   Path where Wikilabels should be installed (example: 
'/vagrant/srv/wikilabels').
 #
 # [*db_name*]
-#   Logical MySQL database name (example: 'wikilabels').
+#   Logical PostgreSQL database name (example: 'wikilabels').
 #
 # [*db_user*]
-#   MySQL user to use to connect to the database (example: 'wikidb').
+#   PostgreSQL user to use to connect to the database (example: 'wikidb').
 #
 # [*db_pass*]
-#   Password for MySQL account (example: 'secret123').
+#   Password for PostgreSQL account (example: 'secret123').
 #
 # [*vhost_name*]
 #   Hostname of the Wikilabels server (example: 
'wikilabels.local.wmftest.net').
@@ -82,7 +82,7 @@
 exec { 'initialize wikilabels database':
 # puppet does not allow specifying separate users for command and 
unless so sudoing ensues
 command => "echo y | sudo -u www-data ${deploy_dir}/bin/wikilabels 
load_schema --reload-test-data",
-unless  => "sudo -u postgres psql -d ${db_name} -c \"SELECT 
'campaign'::regclass\"",
+unless  => "sudo -u postgres psql -d ${db_name} -c \"SELECT 
'campaign'::regclass\" >& /dev/null",
 cwd => $repo_dir,
 require => [
   Exec['create wikilabels database'],

-- 
To view, visit https://gerrit.wikimedia.org/r/405035
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1182e76d094eca9a2ea3ede425d32e349b3e4132
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/dumps[master]: Tolerate empty tables

2018-01-18 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404987 )

Change subject: Tolerate empty tables
..

Tolerate empty tables

This is an edge case encountered in empty or new-ish wikis.

Change-Id: Ibd97ea1086030fe5753abf9fbe28511026669384
---
M xmldumps-backup/dumps/utils.py
1 file changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/dumps 
refs/changes/87/404987/1

diff --git a/xmldumps-backup/dumps/utils.py b/xmldumps-backup/dumps/utils.py
index 86d285e..830b1ee 100644
--- a/xmldumps-backup/dumps/utils.py
+++ b/xmldumps-backup/dumps/utils.py
@@ -482,7 +482,7 @@
 return 1
 
 lines = results.splitlines()
-if lines and lines[1]:
+if lines and lines[1] and lines[1] != 'NULL':
 self.total_pages = int(lines[1])
 
 query = "select MAX(rev_id) from %srevision;" % 
self.db_server_info.db_table_prefix
@@ -497,7 +497,7 @@
 return 1
 
 lines = results.splitlines()
-if lines and lines[1]:
+if lines and lines[1] and lines[1] != 'NULL':
 self.total_edits = int(lines[1])
 
 query = "select MAX(log_id) from %slogging;" % 
self.db_server_info.db_table_prefix
@@ -512,7 +512,7 @@
 return 1
 
 lines = results.splitlines()
-if lines and lines[1]:
+if lines and lines[1] and lines[1] != 'NULL':
 self.total_logitems = int(lines[1])
 
 return 0

-- 
To view, visit https://gerrit.wikimedia.org/r/404987
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibd97ea1086030fe5753abf9fbe28511026669384
Gerrit-PatchSet: 1
Gerrit-Project: operations/dumps
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/dumps[master]: Use a global mysql user if configured

2018-01-18 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404986 )

Change subject: Use a global mysql user if configured
..

Use a global mysql user if configured

This makes it easier to run under vagrant, where PrivateSettings.php doesn't
exist.

Change-Id: I44beb87f44cf26505107e199ba46123700872ee0
---
M xmldumps-backup/dumps/WikiDump.py
1 file changed, 7 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/dumps 
refs/changes/86/404986/1

diff --git a/xmldumps-backup/dumps/WikiDump.py 
b/xmldumps-backup/dumps/WikiDump.py
index 77b86f0..cd5c33d 100644
--- a/xmldumps-backup/dumps/WikiDump.py
+++ b/xmldumps-backup/dumps/WikiDump.py
@@ -76,10 +76,17 @@
 default_dbpassword = ""
 
 if not conf.has_option("wiki", "adminsettings"):
+# Fall back to global db user configuration if available.
+if conf.has_option("database", "user"):
+if conf.has_option("database", "password"):
+return conf.get("database", "user"), conf.get("database", 
"password")
+
+# Fall back to defaults.
 db_user = default_dbuser
 db_password = default_dbpassword
 return db_user, db_password
 
+# Read db user from PrivateSettings.php
 adminfhandle = open(os.path.join(wiki_dir, conf.get("wiki", 
"adminsettings")), "r")
 lines = adminfhandle.readlines()
 adminfhandle.close()

-- 
To view, visit https://gerrit.wikimedia.org/r/404986
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I44beb87f44cf26505107e199ba46123700872ee0
Gerrit-PatchSet: 1
Gerrit-Project: operations/dumps
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: [WIP] Take advantage of MaintenanceBaseTestCase

2018-01-17 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404677 )

Change subject: [WIP] Take advantage of MaintenanceBaseTestCase
..

[WIP] Take advantage of MaintenanceBaseTestCase

Change-Id: I9581adc5141873da201ee5b86914e7119179ce81
---
M tests/phpunit/maintenance/PopulateDatabaseTest.php
M tests/phpunit/maintenance/PurgeScoreCacheTest.php
2 files changed, 24 insertions(+), 18 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/77/404677/1

diff --git a/tests/phpunit/maintenance/PopulateDatabaseTest.php 
b/tests/phpunit/maintenance/PopulateDatabaseTest.php
index 402f331..d7d5f81 100644
--- a/tests/phpunit/maintenance/PopulateDatabaseTest.php
+++ b/tests/phpunit/maintenance/PopulateDatabaseTest.php
@@ -2,6 +2,7 @@
 
 namespace ORES\Tests\Maintenance;
 
+use MediaWiki\Tests\Maintenance\MaintenanceBaseTestCase;
 use ORES\Maintenance\PopulateDatabase;
 
 use ORES\Tests\MockOresServiceBuilder;
@@ -12,7 +13,11 @@
  * @group Database
  * @covers ORES\Maintenance\PopulateDatabase
  */
-class PopulateDatabaseTest extends \MediaWikiTestCase {
+class PopulateDatabaseTest extends MaintenanceBaseTestCase {
+
+   public function getMaintenanceClass() {
+   return PopulateDatabase::class;
+   }
 
public function setUp() {
parent::setUp();
@@ -21,8 +26,6 @@
'ores_model',
'recentchanges',
];
-
-   $this->maintenance = new PopulateDatabase();
 
TestHelper::clearOresTables();
TestHelper::insertModelData();
@@ -158,9 +161,6 @@
foreach ( $oresContents as $revId => $scores ) {
TestHelper::insertOresData( $revId, $scores );
}
-
-   // FIXME: Write a base maintenance test case that suppresses 
output.
-   $argv = array_merge( $argv, [ '--quiet' ] );
 
$this->maintenance->loadWithArgv( $argv );
$this->maintenance->execute();
diff --git a/tests/phpunit/maintenance/PurgeScoreCacheTest.php 
b/tests/phpunit/maintenance/PurgeScoreCacheTest.php
index 3dafa78..478e2ed 100644
--- a/tests/phpunit/maintenance/PurgeScoreCacheTest.php
+++ b/tests/phpunit/maintenance/PurgeScoreCacheTest.php
@@ -3,6 +3,7 @@
 namespace ORES\Tests\Maintenance;
 
 use MediaWiki\MediaWikiServices;
+use MediaWiki\Tests\Maintenance\MaintenanceBaseTestCase;
 
 use ORES\Maintenance\PurgeScoreCache;
 
@@ -25,7 +26,11 @@
  * @group Database
  * @covers ORES\Maintenance\PurgeScoreCache
  */
-class PurgeScoreCacheTest extends \MediaWikiTestCase {
+class PurgeScoreCacheTest extends MaintenanceBaseTestCase {
+
+   public function getMaintenanceClass() {
+   return PurgeScoreCache::class;
+   }
 
public function setUp() {
parent::setUp();
@@ -33,8 +38,6 @@
'ores_classification',
'ores_model',
];
-
-   $this->maintenance = new PurgeScoreCache();
 
TestHelper::clearOresTables();
TestHelper::insertModelData();
@@ -45,9 +48,6 @@
 
public function testPurgeScoreCache_emptyDb() {
TestHelper::clearOresTables();
-
-   // FIXME: Shouldn't be necessary once we capture output.
-   $this->maintenance->loadWithArgv( [ '--quiet' ] );
 
$this->maintenance->execute();
 
@@ -62,7 +62,7 @@
'damaging' => 0.1,
] );
 
-   $this->maintenance->loadWithArgv( [ '--quiet', '--model', 
'not_a_thing' ] );
+   $this->maintenance->loadWithArgv( [ '--model', 'not_a_thing' ] 
);
 
$this->maintenance->execute();
 
@@ -79,6 +79,8 @@
'oresc_probability' => '0.100',
'oresc_model' => (string)TestHelper::DAMAGING,
] ], iterator_to_array( $remainingScores, false ) );
+
+   $this->expectOutputRegex( '/skipping \'not_a_thing\' model/' );
}
 
public function testPurgeScoreCache_all() {
@@ -88,7 +90,7 @@
'damaging' => 0.1,
] );
 
-   $this->maintenance->loadWithArgv( [ '--quiet', '--all' ] );
+   $this->maintenance->loadWithArgv( [ '--all' ] );
 
$this->maintenance->execute();
 
@@ -100,6 +102,10 @@
);
 
$this->assertEquals( [], iterator_to_array( $remainingScores, 
false ) );
+
+   $pattern = '/skipping \'reverted\'.+'
+   . 'purging scores from all model versions from 
\'damaging\'/s';
+   $this->expectOutputRegex( $pattern );
}
 
public function testPurgeScoreCache_oldModels() {
@@ -108,8 +114,6 @@
TestHelper::DAMAGING_OLD => 0.2,

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Replace some of MaintenanceTest with dataProviders

2018-01-17 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404665 )

Change subject: Replace some of MaintenanceTest with dataProviders
..

Replace some of MaintenanceTest with dataProviders

Bug: T184775
Change-Id: Icbaf3cbcf3c51e5b4cd53d55a5230534e74a0a38
---
M tests/phpunit/maintenance/MaintenanceTest.php
1 file changed, 259 insertions(+), 446 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/65/404665/1

diff --git a/tests/phpunit/maintenance/MaintenanceTest.php 
b/tests/phpunit/maintenance/MaintenanceTest.php
index 198fc81..4828eae 100644
--- a/tests/phpunit/maintenance/MaintenanceTest.php
+++ b/tests/phpunit/maintenance/MaintenanceTest.php
@@ -36,459 +36,272 @@
// test.*Intermittent.* tests), the objective of these tests is not to 
describe
// consistent behavior, but rather currently existing behavior.
 
-   function testOutputEmpty() {
-   $this->maintenance->output( "" );
-   $this->assertOutputPrePostShutdown( "", false );
+   /**
+* @dataProvider provideOutputData
+*/
+   function testOutput( $outputs, $expected, $extraNL ) {
+   foreach ( $outputs as $data ) {
+   if ( is_array( $data ) ) {
+   list( $msg, $channel ) = $data;
+   } else {
+   $msg = $data;
+   $channel = null;
+   }
+   $this->maintenance->output( $msg, $channel );
+   }
+   $this->assertOutputPrePostShutdown( $expected, $extraNL );
}
 
-   function testOutputString() {
-   $this->maintenance->output( "foo" );
-   $this->assertOutputPrePostShutdown( "foo", false );
-   }
+   public function provideOutputData() {
+   return [
+   [ [ "" ], "", false ],
+   [ [ "foo" ], "foo", false ],
+   [ [ "foo", "bar" ], "foobar", false ],
+   [ [ "foo\n" ], "foo\n", false ],
+   [ [ "foo\n\n" ], "foo\n\n", false ],
+   [ [ "foo\nbar" ], "foo\nbar", false ],
+   [ [ "foo\nbar\n" ], "foo\nbar\n", false ],
+   [ [ "foo\n", "bar\n" ], "foo\nbar\n", false ],
+   [ [ "", "foo", "", "\n", "ba", "", "r\n" ], 
"foo\nbar\n", false ],
+   [ [ "", "foo", "", "\nb", "a", "", "r\n" ], 
"foo\nbar\n", false ],
+   [ [ [ "foo", "bazChannel" ] ], "foo", true ],
+   [ [ [ "foo\n", "bazChannel" ] ], "foo", true ],
 
-   function testOutputStringString() {
-   $this->maintenance->output( "foo" );
-   $this->maintenance->output( "bar" );
-   $this->assertOutputPrePostShutdown( "foobar", false );
+   // If this test fails, note that output takes strings 
with double line
+   // endings (although output's implementation in this 
situation calls
+   // outputChanneled with a string ending in a nl ... 
which is not allowed
+   // according to the documentation of outputChanneled)
+   [ [ [ "foo\n\n", "bazChannel" ] ], "foo\n", true ],
+   [ [ [ "foo\nbar", "bazChannel" ] ], "foo\nbar", true ],
+   [ [ [ "foo\nbar\n", "bazChannel" ] ], "foo\nbar", true 
],
+   [
+   [
+   [ "foo\n", "bazChannel" ],
+   [ "bar\n", "bazChannel" ],
+   ],
+   "foobar",
+   true
+   ],
+   [
+   [
+   [ "", "bazChannel" ],
+   [ "foo", "bazChannel" ],
+   [ "", "bazChannel" ],
+   [ "\n", "bazChannel" ],
+   [ "ba", "bazChannel" ],
+   [ "", "bazChannel" ],
+   [ "r\n", "bazChannel" ],
+   ],
+   "foobar",
+   true
+   ],
+   [
+   [
+   [ "", "bazChannel" ],
+   [ "foo", "bazChannel" ],
+   [ "", "bazChannel" ],
+   [ "\nb", "bazChannel" ],
+   [ "a", "bazChannel" ],
+

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Refactor MaintenanceTest to use a base test case

2018-01-16 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404606 )

Change subject: Refactor MaintenanceTest to use a base test case
..

Refactor MaintenanceTest to use a base test case

Bug: T184775
Change-Id: I0cb2320f765e847a59c721918401da07a3a2b24a
---
M tests/common/TestsAutoLoader.php
A tests/phpunit/maintenance/MaintenanceBaseTestCase.php
D tests/phpunit/maintenance/MaintenanceFixup.php
M tests/phpunit/maintenance/MaintenanceTest.php
4 files changed, 386 insertions(+), 684 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/06/404606/1

diff --git a/tests/common/TestsAutoLoader.php b/tests/common/TestsAutoLoader.php
index 825cee0..48f6267 100644
--- a/tests/common/TestsAutoLoader.php
+++ b/tests/common/TestsAutoLoader.php
@@ -151,6 +151,7 @@
 
# tests/phpunit/maintenance
'MediaWiki\Tests\Maintenance\DumpTestCase' => 
"$testDir/phpunit/maintenance/DumpTestCase.php",
+   'MediaWiki\Tests\Maintenance\MaintenanceBaseTestCase' => 
"$testDir/phpunit/maintenance/MaintenanceBaseTestCase.php",
'MediaWiki\Tests\Maintenance\MaintenanceFixup' => 
"$testDir/phpunit/maintenance/MaintenanceFixup.php",
 
# tests/phpunit/media
diff --git a/tests/phpunit/maintenance/MaintenanceBaseTestCase.php 
b/tests/phpunit/maintenance/MaintenanceBaseTestCase.php
new file mode 100644
index 000..9474441
--- /dev/null
+++ b/tests/phpunit/maintenance/MaintenanceBaseTestCase.php
@@ -0,0 +1,50 @@
+maintenance ) {
+   $this->maintenance->outputChanneled( false );
+   }
+   parent::tearDown();
+   }
+
+   /**
+* Asserts the output before and after simulating shutdown
+*
+* This function simulates shutdown of self::maintenance.
+*
+* @param string $preShutdownOutput Expected output before simulating 
shutdown
+* @param bool $expectNLAppending Whether or not shutdown simulation is 
expected
+*   to add a newline to the output. If false, $preShutdownOutput is the
+*   expected output after shutdown simulation. Otherwise,
+*   $preShutdownOutput with an appended newline is the expected output
+*   after shutdown simulation.
+*/
+   protected function assertOutputPrePostShutdown( $preShutdownOutput, 
$expectNLAppending ) {
+   $this->assertEquals( $preShutdownOutput, 
$this->getActualOutput(),
+   "Output before shutdown simulation" );
+
+   $this->maintenance->outputChanneled( false );
+
+   $postShutdownOutput = $preShutdownOutput . ( $expectNLAppending 
? "\n" : "" );
+   $this->expectOutputString( $postShutdownOutput );
+   }
+
+}
+
diff --git a/tests/phpunit/maintenance/MaintenanceFixup.php 
b/tests/phpunit/maintenance/MaintenanceFixup.php
deleted file mode 100644
index 40da42d..000
--- a/tests/phpunit/maintenance/MaintenanceFixup.php
+++ /dev/null
@@ -1,142 +0,0 @@
-shutdownSimulated ) {
-   $this->testCase->fail( __METHOD__ . " called more than 
once" );
-   }
-
-   // The cleanup action.
-   $this->outputChanneled( false );
-
-   // Bookkeeping that we simulated the clean up.
-   $this->shutdownSimulated = true;
-   }
-
-   // Note that the "public" here does not change visibility
-   public function outputChanneled( $msg, $channel = null ) {
-   if ( $this->shutdownSimulated ) {
-   if ( $msg !== false ) {
-   $this->testCase->fail( "Already past simulated 
shutdown, but msg is "
-   . "not false. Did the hack in 
Maintenance.php change? Please "
-   . "adapt the test case or 
Maintenance.php" );
-   }
-
-   // The current call is the one registered via 
register_shutdown_function.
-   // We can safely ignore it, as we simulated this one 
via simulateShutdown
-   // before (if we did not, the destructor of this 
instance will warn about
-   // it)
-   return;
-   }
-
-   call_user_func_array( [ "parent", __FUNCTION__ ], 
func_get_args() );
-   }
-
-   /**
-* Safety net around register_shutdown_function of Maintenance.php
-*/
-   public function __destruct() {
-   if ( !$this->shutdownSimulated ) {
-   // Someone generated a MaintenanceFixup instance 
without calling
-   // simulateShutdown. We'd have to raise a PHPUnit 
exception to correctly
-   // flag this illegal usage. However, we are already in 
a destruktor, which
-   // would trigger undefined behavior. Hence,

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Revert accidentally removed trait

2018-01-16 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404605 )

Change subject: Revert accidentally removed trait
..

Revert accidentally removed trait

Change-Id: I82283be89f3f155d7793d4c900927d81a60d5e95
---
M tests/phpunit/maintenance/BenchmarkerTest.php
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/05/404605/1

diff --git a/tests/phpunit/maintenance/BenchmarkerTest.php 
b/tests/phpunit/maintenance/BenchmarkerTest.php
index 83e39cb..765884c 100644
--- a/tests/phpunit/maintenance/BenchmarkerTest.php
+++ b/tests/phpunit/maintenance/BenchmarkerTest.php
@@ -11,6 +11,8 @@
  */
 class BenchmarkerTest extends PHPUnit_Framework_TestCase {
 
+   use MediaWikiCoversValidator;
+
public function testBenchSimple() {
$bench = $this->getMockBuilder( Benchmarker::class )
->setMethods( [ 'execute', 'output' ] )

-- 
To view, visit https://gerrit.wikimedia.org/r/404605
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I82283be89f3f155d7793d4c900927d81a60d5e95
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Use absolute scope for systemd template variables

2018-01-16 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404521 )

Change subject: Use absolute scope for systemd template variables
..

Use absolute scope for systemd template variables

Bug: T184077
Change-Id: I123ba4d1b19c4e5c2c51f9b042e3ac06ffae3362
---
M puppet/modules/ores/templates/systemd/ores-celery.erb
M puppet/modules/ores/templates/systemd/ores-wsgi.erb
M puppet/modules/wikilabels/manifests/init.pp
M puppet/modules/wikilabels/templates/systemd/wikilabels.erb
4 files changed, 7 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/21/404521/1

diff --git a/puppet/modules/ores/templates/systemd/ores-celery.erb 
b/puppet/modules/ores/templates/systemd/ores-celery.erb
index 21493e8..bc0c196 100644
--- a/puppet/modules/ores/templates/systemd/ores-celery.erb
+++ b/puppet/modules/ores/templates/systemd/ores-celery.erb
@@ -11,8 +11,8 @@
 User=www-data
 Group=www-data
 SyslogIdentifier=ores-celery
-WorkingDirectory=<%= @ores_root %>
-ExecStart=<%= @venv_dir %>/bin/ores applications.celery --logging-config <%= 
@logging_config %>
+WorkingDirectory=<%= scope['::ores::ores_root'] %>
+ExecStart=<%= scope['::ores::venv_dir'] %>/bin/ores applications.celery 
--logging-config <%= scope['::ores::logging_config'] %>
 Restart=always
 
 [Install]
diff --git a/puppet/modules/ores/templates/systemd/ores-wsgi.erb 
b/puppet/modules/ores/templates/systemd/ores-wsgi.erb
index afeb81b..97bfab6 100644
--- a/puppet/modules/ores/templates/systemd/ores-wsgi.erb
+++ b/puppet/modules/ores/templates/systemd/ores-wsgi.erb
@@ -11,8 +11,8 @@
 User=www-data
 Group=www-data
 SyslogIdentifier=ores-wsgi
-WorkingDirectory=<%= @ores_root %>
-ExecStart=<%= @venv_dir %>/bin/ores applications.wsgi --port=<%= @port %> 
--processes=1 --logging-config <%= @logging_config %>
+WorkingDirectory=<%= scope['::ores::ores_root'] %>
+ExecStart=<%= scope['::ores::venv_dir'] %>/bin/ores applications.wsgi 
--port=<%= scope['::ores::port'] %> --processes=1 --logging-config <%= 
scope['::ores::logging_config'] %>
 Restart=always
 
 [Install]
diff --git a/puppet/modules/wikilabels/manifests/init.pp 
b/puppet/modules/wikilabels/manifests/init.pp
index 9c742b2..fb8ee4c 100644
--- a/puppet/modules/wikilabels/manifests/init.pp
+++ b/puppet/modules/wikilabels/manifests/init.pp
@@ -84,7 +84,7 @@
 command => "echo y | sudo -u www-data ${deploy_dir}/bin/wikilabels 
load_schema --reload-test-data",
 unless  => "sudo -u postgres psql -d ${db_name} -c \"SELECT 
'campaign'::regclass\"",
 cwd => $repo_dir,
-require =>[
+require => [
   Exec['create wikilabels database'],
   Virtualenv::Package['wikilabels'],
   File[$cfg_file],
diff --git a/puppet/modules/wikilabels/templates/systemd/wikilabels.erb 
b/puppet/modules/wikilabels/templates/systemd/wikilabels.erb
index e5c651d..8b3fdeb 100644
--- a/puppet/modules/wikilabels/templates/systemd/wikilabels.erb
+++ b/puppet/modules/wikilabels/templates/systemd/wikilabels.erb
@@ -11,8 +11,8 @@
 User=www-data
 Group=www-data
 SyslogIdentifier=wikilabels
-WorkingDirectory=<%= @repo_dir %>
-ExecStart=<%= @deploy_dir %>/bin/wikilabels dev_server --port=<%= @port %>
+WorkingDirectory=<%= scope['::wikilabels::repo_dir'] %>
+ExecStart=<%= scope['::wikilabels::deploy_dir'] %>/bin/wikilabels dev_server 
--port=<%= scope['::wikilabels::port'] %>
 Restart=always
 
 [Install]

-- 
To view, visit https://gerrit.wikimedia.org/r/404521
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I123ba4d1b19c4e5c2c51f9b042e3ac06ffae3362
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: New namespace MediaWiki\Tests\Maintenance

2018-01-16 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404512 )

Change subject: New namespace MediaWiki\Tests\Maintenance
..

New namespace MediaWiki\Tests\Maintenance

Bug: T184775
Change-Id: I041b8eb4f1ef3cf882a8082ff3f208604b8edf96
---
M tests/common/TestsAutoLoader.php
M tests/phpunit/maintenance/BenchmarkerTest.php
M tests/phpunit/maintenance/DumpTestCase.php
M tests/phpunit/maintenance/MaintenanceTest.php
M tests/phpunit/maintenance/backupPrefetchTest.php
M tests/phpunit/maintenance/backupTextPassTest.php
M tests/phpunit/maintenance/backup_LogTest.php
M tests/phpunit/maintenance/backup_PageTest.php
M tests/phpunit/maintenance/categoriesRdfTest.php
M tests/phpunit/maintenance/fetchTextTest.php
10 files changed, 69 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/12/404512/1

diff --git a/tests/common/TestsAutoLoader.php b/tests/common/TestsAutoLoader.php
index a51008e..1c41a6d 100644
--- a/tests/common/TestsAutoLoader.php
+++ b/tests/common/TestsAutoLoader.php
@@ -150,7 +150,7 @@
'GenericArrayObjectTest' => 
"$testDir/phpunit/includes/libs/GenericArrayObjectTest.php",
 
# tests/phpunit/maintenance
-   'DumpTestCase' => "$testDir/phpunit/maintenance/DumpTestCase.php",
+   'MediaWiki\Tests\Maintenance\DumpTestCase' => 
"$testDir/phpunit/maintenance/DumpTestCase.php",
 
# tests/phpunit/media
'FakeDimensionFile' => 
"$testDir/phpunit/includes/media/FakeDimensionFile.php",
diff --git a/tests/phpunit/maintenance/BenchmarkerTest.php 
b/tests/phpunit/maintenance/BenchmarkerTest.php
index ec570e2..67184a2 100644
--- a/tests/phpunit/maintenance/BenchmarkerTest.php
+++ b/tests/phpunit/maintenance/BenchmarkerTest.php
@@ -1,13 +1,16 @@
 getMockBuilder( Benchmarker::class )
diff --git a/tests/phpunit/maintenance/DumpTestCase.php 
b/tests/phpunit/maintenance/DumpTestCase.php
index 58cb6f3..c872993 100644
--- a/tests/phpunit/maintenance/DumpTestCase.php
+++ b/tests/phpunit/maintenance/DumpTestCase.php
@@ -1,5 +1,14 @@
 tablesUsed[] = 'text';
 
$this->mergeMwGlobalArrayValue( 'wgContentHandlers', [
-   "BackupTextPassTestModel" => 
"BackupTextPassTestModelHandler"
+   "BackupTextPassTestModel" => 
'MediaWiki\Tests\Maintenance\BackupTextPassTestModelHandler',
] );
 
$ns = $this->getDefaultWikitextNS();
diff --git a/tests/phpunit/maintenance/backup_LogTest.php 
b/tests/phpunit/maintenance/backup_LogTest.php
index 91e1b1e..c215b99 100644
--- a/tests/phpunit/maintenance/backup_LogTest.php
+++ b/tests/phpunit/maintenance/backup_LogTest.php
@@ -1,4 +1,13 @@
 https://gerrit.wikimedia.org/r/404512
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I041b8eb4f1ef3cf882a8082ff3f208604b8edf96
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: [WIP] New namespace for tests/phpunit/maintenance, begin spl...

2018-01-16 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404490 )

Change subject: [WIP] New namespace for tests/phpunit/maintenance, begin 
splitting out maintenance test base classes
..

[WIP] New namespace for tests/phpunit/maintenance, begin splitting out 
maintenance test base classes

Bug: T184775
Change-Id: I73c2f3c6975deec50cf879201cf292c217b51c51
---
M tests/common/TestsAutoLoader.php
M tests/phpunit/maintenance/DumpTestCase.php
A tests/phpunit/maintenance/MaintenanceFixup.php
M tests/phpunit/maintenance/MaintenanceTest.php
M tests/phpunit/maintenance/backupTextPassTest.php
M tests/phpunit/maintenance/backup_LogTest.php
M tests/phpunit/maintenance/backup_PageTest.php
7 files changed, 186 insertions(+), 139 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/90/404490/1

diff --git a/tests/common/TestsAutoLoader.php b/tests/common/TestsAutoLoader.php
index a51008e..825cee0 100644
--- a/tests/common/TestsAutoLoader.php
+++ b/tests/common/TestsAutoLoader.php
@@ -150,7 +150,8 @@
'GenericArrayObjectTest' => 
"$testDir/phpunit/includes/libs/GenericArrayObjectTest.php",
 
# tests/phpunit/maintenance
-   'DumpTestCase' => "$testDir/phpunit/maintenance/DumpTestCase.php",
+   'MediaWiki\Tests\Maintenance\DumpTestCase' => 
"$testDir/phpunit/maintenance/DumpTestCase.php",
+   'MediaWiki\Tests\Maintenance\MaintenanceFixup' => 
"$testDir/phpunit/maintenance/MaintenanceFixup.php",
 
# tests/phpunit/media
'FakeDimensionFile' => 
"$testDir/phpunit/includes/media/FakeDimensionFile.php",
diff --git a/tests/phpunit/maintenance/DumpTestCase.php 
b/tests/phpunit/maintenance/DumpTestCase.php
index 58cb6f3..c872993 100644
--- a/tests/phpunit/maintenance/DumpTestCase.php
+++ b/tests/phpunit/maintenance/DumpTestCase.php
@@ -1,5 +1,14 @@
 shutdownSimulated ) {
+   $this->testCase->fail( __METHOD__ . " called more than 
once" );
+   }
+
+   // The cleanup action.
+   $this->outputChanneled( false );
+
+   // Bookkeeping that we simulated the clean up.
+   $this->shutdownSimulated = true;
+   }
+
+   // Note that the "public" here does not change visibility
+   public function outputChanneled( $msg, $channel = null ) {
+   if ( $this->shutdownSimulated ) {
+   if ( $msg !== false ) {
+   $this->testCase->fail( "Already past simulated 
shutdown, but msg is "
+   . "not false. Did the hack in 
Maintenance.php change? Please "
+   . "adapt the test case or 
Maintenance.php" );
+   }
+
+   // The current call is the one registered via 
register_shutdown_function.
+   // We can safely ignore it, as we simulated this one 
via simulateShutdown
+   // before (if we did not, the destructor of this 
instance will warn about
+   // it)
+   return;
+   }
+
+   call_user_func_array( [ "parent", __FUNCTION__ ], 
func_get_args() );
+   }
+
+   /**
+* Safety net around register_shutdown_function of Maintenance.php
+*/
+   public function __destruct() {
+   if ( !$this->shutdownSimulated ) {
+   // Someone generated a MaintenanceFixup instance 
without calling
+   // simulateShutdown. We'd have to raise a PHPUnit 
exception to correctly
+   // flag this illegal usage. However, we are already in 
a destruktor, which
+   // would trigger undefined behavior. Hence, we can only 
report to the
+   // error output :( Hopefully people read the PHPUnit 
output.
+   $name = $this->testCase->getName();
+   fwrite( STDERR, "ERROR! Instance of " . __CLASS__ . " 
for test $name "
+   . "destructed without calling simulateShutdown 
method. Call "
+   . "simulateShutdown on the instance before it 
gets destructed." );
+   }
+
+   // The following guard is required, as PHP does not offer 
default destructors :(
+   if ( is_callable( "parent::__destruct" ) ) {
+   parent::__destruct();
+   }
+   }
+
+   public function __construct( MediaWikiTestCase $testCase ) {
+   parent::__construct();
+   $this->testCase = $testCase;
+   }
+
+   // --- Making protected functions visible for test
+
+   public function output( $out, $channel = null ) {
+   // Just to make PHP not nag about signature mismatches, we 
copied
+   // Maintenance::output signature. However, we do not

[MediaWiki-commits] [Gerrit] mediawiki/core[master]: Minor documentation cleanup

2018-01-16 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404489 )

Change subject: Minor documentation cleanup
..

Minor documentation cleanup

Bug: T184775
Change-Id: I883a2ebc0a1ce7667a92623a4861be80df26a65d
---
M tests/phpunit/maintenance/MaintenanceTest.php
1 file changed, 16 insertions(+), 14 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/core 
refs/changes/89/404489/1

diff --git a/tests/phpunit/maintenance/MaintenanceTest.php 
b/tests/phpunit/maintenance/MaintenanceTest.php
index e21e726..253327b 100644
--- a/tests/phpunit/maintenance/MaintenanceTest.php
+++ b/tests/phpunit/maintenance/MaintenanceTest.php
@@ -1,29 +1,31 @@
 https://gerrit.wikimedia.org/r/404489
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I883a2ebc0a1ce7667a92623a4861be80df26a65d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/core
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Move mock ORESService into a new helper class

2018-01-16 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404473 )

Change subject: Move mock ORESService into a new helper class
..

Move mock ORESService into a new helper class

Bug: T184142
Change-Id: Ia88360f068d873da8a3cff930011ec2de7abf259
---
A tests/phpunit/includes/MockOresServiceBuilder.php
M tests/phpunit/includes/ScoreFetcherTest.php
2 files changed, 53 insertions(+), 42 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/73/404473/1

diff --git a/tests/phpunit/includes/MockOresServiceBuilder.php 
b/tests/phpunit/includes/MockOresServiceBuilder.php
new file mode 100644
index 000..69e1f26
--- /dev/null
+++ b/tests/phpunit/includes/MockOresServiceBuilder.php
@@ -0,0 +1,51 @@
+getMockBuilder( ORESService::class )
+   ->disableOriginalConstructor()
+   ->getMock();
+
+   $mock->expects( $test->any() )
+   ->method( 'request' )
+   ->willReturnCallback( [ self::class, 'mockORESResponse' 
] );
+
+   return $mock;
+   }
+
+   public static function mockORESResponse( array $params, 
$originalRequest = null ) {
+   $models = [];
+   foreach ( explode( '|', $params['models'] ) as $model ) {
+   $models[$model] = [ 'version' => '0.0.4' ];
+   }
+
+   $scores = [];
+   foreach ( explode( '|', $params['revids'] ) as $revid ) {
+   $scores[(string)$revid] = self::mockRevisionResponse( 
$revid, array_keys( $models ) );
+   }
+
+   return [ ORESService::getWikiID() => [ 'models' => $models, 
'scores' => $scores ] ];
+   }
+
+   public static function mockRevisionResponse( $revid, $models ) {
+   $result = [];
+   foreach ( $models as $model ) {
+   $result[$model] = [ 'score' => [] ];
+   $probability = (float)strrev( substr( $revid, -2 ) ) / 
100;
+   $result[$model]['score']['probability'] = [
+   'true' => $probability,
+   'false' => 1 - $probability
+   ];
+   $result[$model]['score']['prediction'] = $probability > 
0.5;
+   }
+   return $result;
+   }
+
+}
diff --git a/tests/phpunit/includes/ScoreFetcherTest.php 
b/tests/phpunit/includes/ScoreFetcherTest.php
index 095b9dc..2aaabd9 100644
--- a/tests/phpunit/includes/ScoreFetcherTest.php
+++ b/tests/phpunit/includes/ScoreFetcherTest.php
@@ -2,7 +2,6 @@
 
 namespace ORES\Tests;
 
-use ORES\ORESService;
 use ORES\ScoreFetcher;
 use ORES\Storage\HashModelLookup;
 
@@ -25,51 +24,12 @@
'goodfaith' => [ 'id' => self::GOODFAITH, 'version' => 
'0.0.3' ],
];
$this->setService( 'ORESModelLookup', new HashModelLookup( 
$modelData ) );
-   $this->setService( 'ORESService', $this->getORESServiceMock() );
+   $mockOresService = MockOresServiceBuilder::getORESServiceMock( 
$this );
+   $this->setService( 'ORESService', $mockOresService );
$this->setMwGlobals( [
'wgOresModels' => [ 'damaging' => true ],
] );
$this->tablesUsed[] = 'ores_model';
-   }
-
-   private function getORESServiceMock() {
-   $mock = $this->getMockBuilder( ORESService::class )
-   ->disableOriginalConstructor()
-   ->getMock();
-
-   $mock->expects( $this->any() )
-   ->method( 'request' )
-   ->willReturnCallback( [ $this, 'mockORESResponse' ] );
-
-   return $mock;
-   }
-
-   public static function mockORESResponse( array $params, 
$originalRequest = null ) {
-   $models = [];
-   foreach ( explode( '|', $params['models'] ) as $model ) {
-   $models[$model] = [ 'version' => '0.0.4' ];
-   }
-
-   $scores = [];
-   foreach ( explode( '|', $params['revids'] ) as $revid ) {
-   $scores[(string)$revid] = self::mockRevisionResponse( 
$revid, array_keys( $models ) );
-   }
-
-   return [ ORESService::getWikiID() => [ 'models' => $models, 
'scores' => $scores ] ];
-   }
-
-   public static function mockRevisionResponse( $revid, $models ) {
-   $result = [];
-   foreach ( $models as $model ) {
-   $result[$model] = [ 'score' => [] ];
-   $probability = (float)strrev( substr( $revid, -2 ) ) / 
100;
-   $result[$model]['score']['probability'] = [
-   'true' => $probability,
-

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Minor test clean up

2018-01-16 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/404474 )

Change subject: Minor test clean up
..

Minor test clean up

Change-Id: I95882a35ceb69d8266c5797d35f466db5f309e88
---
M tests/phpunit/maintenance/PurgeScoreCacheTest.php
1 file changed, 4 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/74/404474/1

diff --git a/tests/phpunit/maintenance/PurgeScoreCacheTest.php 
b/tests/phpunit/maintenance/PurgeScoreCacheTest.php
index 9209716..3dafa78 100644
--- a/tests/phpunit/maintenance/PurgeScoreCacheTest.php
+++ b/tests/phpunit/maintenance/PurgeScoreCacheTest.php
@@ -37,12 +37,15 @@
$this->maintenance = new PurgeScoreCache();
 
TestHelper::clearOresTables();
+   TestHelper::insertModelData();
 
// Reset service to purge cached models.
MediaWikiServices::getInstance()->resetServiceForTesting( 
'ORESModelLookup' );
}
 
-   public function testPurgeScoreCache_noop() {
+   public function testPurgeScoreCache_emptyDb() {
+   TestHelper::clearOresTables();
+
// FIXME: Shouldn't be necessary once we capture output.
$this->maintenance->loadWithArgv( [ '--quiet' ] );
 
@@ -55,7 +58,6 @@
 
public function testPurgeScoreCache_bad_model() {
$revId = mt_rand( 1000,  );
-   TestHelper::insertModelData();
TestHelper::insertOresData( $revId, [
'damaging' => 0.1,
] );
@@ -81,7 +83,6 @@
 
public function testPurgeScoreCache_all() {
$revId = mt_rand( 1000,  );
-   TestHelper::insertModelData();
TestHelper::insertOresData( $revId, [
TestHelper::DAMAGING_OLD => 0.2,
'damaging' => 0.1,
@@ -103,7 +104,6 @@
 
public function testPurgeScoreCache_oldModels() {
$revId = mt_rand( 1000,  );
-   TestHelper::insertModelData();
TestHelper::insertOresData( $revId, [
TestHelper::DAMAGING_OLD => 0.2,
'damaging' => 0.1,
@@ -134,7 +134,6 @@
$revId = mt_rand( 1000,  );
$revIdOld = $revId - 1;
 
-   TestHelper::insertModelData();
TestHelper::insertOresData( $revId, [
'damaging' => 0.1,
] );
@@ -167,7 +166,6 @@
 
public function testPurgeScoreCache_oneModel() {
$revId = mt_rand( 1000,  );
-   TestHelper::insertModelData();
TestHelper::insertOresData( $revId, [
'damaging' => 0.1,
'reverted' => 0.3,

-- 
To view, visit https://gerrit.wikimedia.org/r/404474
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I95882a35ceb69d8266c5797d35f466db5f309e88
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: [WIP] Tests for PopulateDatabase

2018-01-11 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403870 )

Change subject: [WIP] Tests for PopulateDatabase
..

[WIP] Tests for PopulateDatabase

Bug: T184140
Change-Id: I6fec77e85e1cc45e10d855f8edcef2d3e891cf64
---
A tests/phpunit/maintenance/PopulateDatabaseTest.php
M tests/phpunit/maintenance/PurgeScoreCacheTest.php
2 files changed, 363 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/70/403870/1

diff --git a/tests/phpunit/maintenance/PopulateDatabaseTest.php 
b/tests/phpunit/maintenance/PopulateDatabaseTest.php
new file mode 100644
index 000..070138b
--- /dev/null
+++ b/tests/phpunit/maintenance/PopulateDatabaseTest.php
@@ -0,0 +1,359 @@
+tablesUsed = [
+   'ores_classification',
+   'ores_model',
+   'recentchanges',
+   ];
+
+   $this->maintenance = new PopulateDatabase();
+
+   TestHelper::clearOresTables();
+   TestHelper::insertModelData();
+   \wfGetDB( DB_MASTER )->delete( 'recentchanges', '*', __METHOD__ 
);
+
+   $this->setMwGlobals( [
+   'wgOresWikiId' => 'wiki',
+   ] );
+   $this->mockOresService = $this->getMockBuilder( 
ORESService::class )
+   ->disableOriginalConstructor()
+   ->getMock();
+   $this->setService( 'ORESService', $this->mockOresService );
+   }
+
+   public function testPopulateDatabase_single() {
+   \wfGetDB( DB_MASTER )->insert( 'recentchanges', [
+   [ 'rc_this_oldid' => '123', 'rc_user_text' => 
'TestUser' ],
+   ], __METHOD__ );
+   $this->mockOresService->method( 'request' )
+   ->with(
+   $this->equalTo( [
+   'models' => 
'damaging|goodfaith|reverted|wp10',
+   'revids' => '123',
+   ] ),
+   $this->equalTo( null )
+   )->willReturn( [
+   'wiki' => [
+   'models' => [
+   'damaging' => [
+   'version' => '0.1',
+   ],
+   ],
+   'scores' => [
+   '123' => [
+   'damaging' => [
+   'score' => [
+   
'prediction' => false,
+   
'probability' => [
+   
'false' => 0.1,
+   
'true' => 0.9,
+   ],
+   ],
+   ],
+   ],
+   ],
+   ],
+   ] );
+
+   $this->maintenance->loadWithArgv( [ '--quiet' ] );
+   $this->maintenance->execute();
+
+   $scores = \wfGetDB( DB_REPLICA )->select(
+   [ 'ores_classification' ],
+   [ 'oresc_rev', 'oresc_class', 'oresc_probability', 
'oresc_model' ],
+   null,
+   __METHOD__
+   );
+
+   $this->assertEquals( [ (object)[
+   'oresc_rev' => '123',
+   'oresc_class' => '1',
+   'oresc_probability' => '0.900',
+   'oresc_model' => (string)TestHelper::DAMAGING,
+   ] ], iterator_to_array( $scores, false ) );
+   }
+
+   public function testPopulateDatabase_oneExists() {
+   \wfGetDB( DB_MASTER )->insert( 'recentchanges', [
+   [ 'rc_this_oldid' => '123', 'rc_user_text' => 
'TestUser' ],
+   [ 'rc_this_oldid' => '321', 'rc_user_text' => 
'TestUser' ],
+   ], __METHOD__ );
+   TestHelper::insertOresData( 123, [
+   'damaging' => 0.1,
+   ] );
+   $this->mockOresService->method( 'request' )
+   ->with(
+   $this->equalTo( [
+   'models' =>

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Steal model fixtures for TestHelper; add dirty tricks

2018-01-11 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403838 )

Change subject: Steal model fixtures for TestHelper; add dirty tricks
..

Steal model fixtures for TestHelper; add dirty tricks

Tricks allow for insertion by raw ID.

Bug: T184140
Change-Id: I762106adaff40cf6df151f9479704b730c163308
---
M tests/phpunit/includes/Storage/SqlModelLookupTest.php
M tests/phpunit/includes/TestHelper.php
2 files changed, 62 insertions(+), 45 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/38/403838/1

diff --git a/tests/phpunit/includes/Storage/SqlModelLookupTest.php 
b/tests/phpunit/includes/Storage/SqlModelLookupTest.php
index 31f7bf8..c69eabe 100644
--- a/tests/phpunit/includes/Storage/SqlModelLookupTest.php
+++ b/tests/phpunit/includes/Storage/SqlModelLookupTest.php
@@ -14,10 +14,6 @@
  */
 class SqlModelLookupTest extends MediaWikiLangTestCase {
 
-   const DAMAGING_OLD = 1;
-   const REVERTED = 2;
-   const DAMAGING = 3;
-
/**
 * @var SqlModelLookup
 */
@@ -27,52 +23,22 @@
parent::setUp();
 
$this->tablesUsed[] = 'ores_model';
-   self::insertModelData();
+   TestHelper::insertModelData();
$this->storage = new SqlModelLookup( 
MediaWikiServices::getInstance()->getDBLoadBalancer() );
-   }
-
-   public static function insertModelData() {
-   $db = \wfGetDB( DB_MASTER );
-   $dump = [
-   [
-   'oresm_id' => self::DAMAGING,
-   'oresm_name' => 'damaging',
-   'oresm_version' => '0.0.2',
-   'oresm_is_current' => true
-   ],
-   [
-   'oresm_id' => self::REVERTED,
-   'oresm_name' => 'reverted',
-   'oresm_version' => '0.0.1',
-   'oresm_is_current' => true
-   ],
-   [
-   'oresm_id' => self::DAMAGING_OLD,
-   'oresm_name' => 'damaging',
-   'oresm_version' => '0.0.1',
-   'oresm_is_current' => false
-   ],
-   ];
-
-   $db->delete( 'ores_model', '*' );
-
-   foreach ( $dump as $row ) {
-   $db->insert( 'ores_model', $row );
-   }
}
 
public function testGetModels() {
$models = $this->storage->getModels();
$expected = [
-   'reverted' => [ 'id' => self::REVERTED, 'version' => 
'0.0.1' ],
-   'damaging' => [ 'id' => self::DAMAGING, 'version' => 
'0.0.2' ]
+   'reverted' => [ 'id' => TestHelper::REVERTED, 'version' 
=> '0.0.1' ],
+   'damaging' => [ 'id' => TestHelper::DAMAGING, 'version' 
=> '0.0.2' ]
];
$this->assertEquals( $expected, $models );
}
 
public function testGetModelId() {
-   $this->assertEquals( self::REVERTED, 
$this->storage->getModelId( 'reverted' ) );
-   $this->assertEquals( self::DAMAGING, 
$this->storage->getModelId( 'damaging' ) );
+   $this->assertEquals( TestHelper::REVERTED, 
$this->storage->getModelId( 'reverted' ) );
+   $this->assertEquals( TestHelper::DAMAGING, 
$this->storage->getModelId( 'damaging' ) );
}
 
public function testGetInvalidModelId() {
diff --git a/tests/phpunit/includes/TestHelper.php 
b/tests/phpunit/includes/TestHelper.php
index de6455a..ccc5917 100644
--- a/tests/phpunit/includes/TestHelper.php
+++ b/tests/phpunit/includes/TestHelper.php
@@ -12,6 +12,10 @@
 
 class TestHelper {
 
+   const DAMAGING_OLD = 1;
+   const REVERTED = 2;
+   const DAMAGING = 3;
+
public static function getTablesUsed() {
$tablesUsed = [
'recentchanges',
@@ -24,20 +28,67 @@
}
 
public static function clearOresTables() {
-   wfGetDB( DB_MASTER )->delete( 'recentchanges', '*', __METHOD__ 
);
-   wfGetDB( DB_MASTER )->delete( 'ores_model', '*', __METHOD__ );
-   wfGetDB( DB_MASTER )->delete( 'ores_classification', '*', 
__METHOD__ );
+   \wfGetDB( DB_MASTER )->delete( 'recentchanges', '*', __METHOD__ 
);
+   \wfGetDB( DB_MASTER )->delete( 'ores_model', '*', __METHOD__ );
+   \wfGetDB( DB_MASTER )->delete( 'ores_classification', '*', 
__METHOD__ );
}
 
-   public static function insertOresData( Revision $revision, $scores ) {
+   public static function insertModelData() {
+   $db = \wfGetDB(

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Add a maintenance script test

2018-01-11 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403839 )

Change subject: Add a maintenance script test
..

Add a maintenance script test

TODO: See T184775, we could get rid of the --quiet flag, captureu and verify
output.

Bug: T184140
Change-Id: I1deeb809e27c2daddf9f221fb0ea2cbd36aa166b
---
M maintenance/PurgeScoreCache.php
A tests/phpunit/maintenance/PurgeScoreCacheTest.php
2 files changed, 198 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/39/403839/1

diff --git a/maintenance/PurgeScoreCache.php b/maintenance/PurgeScoreCache.php
index b7f2480..357fd7a 100644
--- a/maintenance/PurgeScoreCache.php
+++ b/maintenance/PurgeScoreCache.php
@@ -43,10 +43,10 @@
$description = 'old rows';
} elseif ( $this->hasOption( 'all' ) ) {
$deletedRows = $this->purge( $model, true, 
$this->mBatchSize );
-   $description = 'old model versions';
+   $description = 'scores from all model versions';
} else {
$deletedRows = $this->purge( $model, false, 
$this->mBatchSize );
-   $description = 'all rows';
+   $description = 'scores from old model versions';
}
if ( $deletedRows ) {
$this->output( "   ...purging $description from 
'$model' model': deleted $deletedRows rows\n" );
@@ -120,7 +120,7 @@
array $tables,
array $conditions,
array $join_conds,
-   $batchSize = 1000
+   $batchSize
) {
$dbr = \wfGetDB( DB_REPLICA );
$dbw = \wfGetDB( DB_MASTER );
diff --git a/tests/phpunit/maintenance/PurgeScoreCacheTest.php 
b/tests/phpunit/maintenance/PurgeScoreCacheTest.php
new file mode 100644
index 000..9209716
--- /dev/null
+++ b/tests/phpunit/maintenance/PurgeScoreCacheTest.php
@@ -0,0 +1,195 @@
+https://phabricator.wikimedia.org/T184775
+ *
+ * This ficitious test class would have to be autoloadable, otherwise:
+ *
+ * require_once getenv( 'MW_INSTALL_PATH' ) !== false
+ * ? getenv( 'MW_INSTALL_PATH' ) . 
'/tests/phpunit/maintenance/MaintenanceTest.php'
+ * : __DIR__ . 
'/../../../../../tests/phpunit/maintenance/MaintenanceTest.php';
+ */
+
+/**
+ * @group ORES
+ * @group Database
+ * @covers ORES\Maintenance\PurgeScoreCache
+ */
+class PurgeScoreCacheTest extends \MediaWikiTestCase {
+
+   public function setUp() {
+   parent::setUp();
+   $this->tablesUsed = [
+   'ores_classification',
+   'ores_model',
+   ];
+
+   $this->maintenance = new PurgeScoreCache();
+
+   TestHelper::clearOresTables();
+
+   // Reset service to purge cached models.
+   MediaWikiServices::getInstance()->resetServiceForTesting( 
'ORESModelLookup' );
+   }
+
+   public function testPurgeScoreCache_noop() {
+   // FIXME: Shouldn't be necessary once we capture output.
+   $this->maintenance->loadWithArgv( [ '--quiet' ] );
+
+   $this->maintenance->execute();
+
+   // Well, this is dirty but the point I want to demonstrate is 
that
+   // the previous function didn't crash.
+   $this->assertTrue( true );
+   }
+
+   public function testPurgeScoreCache_bad_model() {
+   $revId = mt_rand( 1000,  );
+   TestHelper::insertModelData();
+   TestHelper::insertOresData( $revId, [
+   'damaging' => 0.1,
+   ] );
+
+   $this->maintenance->loadWithArgv( [ '--quiet', '--model', 
'not_a_thing' ] );
+
+   $this->maintenance->execute();
+
+   $remainingScores = \wfGetDB( DB_REPLICA )->select(
+   [ 'ores_classification' ],
+   [ 'oresc_rev', 'oresc_class', 'oresc_probability', 
'oresc_model' ],
+   [ 'oresc_rev' => $revId ],
+   __METHOD__
+   );
+
+   $this->assertEquals( [ (object)[
+   'oresc_rev' => (string)$revId,
+   'oresc_class' => '1',
+   'oresc_probability' => '0.100',
+   'oresc_model' => (string)TestHelper::DAMAGING,
+   ] ], iterator_to_array( $remainingScores, false ) );
+   }
+
+   public function testPurgeScoreCache_all() {
+   $revId = mt_rand( 1000,  );
+   TestHelper::insertModelData();
+   TestHelper::insertOresData( $revId, [
+   TestHelper::DAMAGING_OLD =>

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Namespace maintenance scripts so they're discoverable from t...

2018-01-11 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403663 )

Change subject: Namespace maintenance scripts so they're discoverable from tests
..

Namespace maintenance scripts so they're discoverable from tests

Bug: T184140
Change-Id: Ia6e82b19bac1d4a7824a94a51cac1e0cb750bbde
---
M extension.json
M maintenance/CleanDuplicateScores.php
M maintenance/DumpThresholds.php
M maintenance/PopulateDatabase.php
M maintenance/PurgeScoreCache.php
5 files changed, 7 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/63/403663/1

diff --git a/extension.json b/extension.json
index d7f8e5d..b0a7a6b 100644
--- a/extension.json
+++ b/extension.json
@@ -13,6 +13,7 @@
},
"AutoloadNamespaces": {
"ORES\\": "includes/",
+   "ORES\\Maintenance\\": "maintenance/",
"ORES\\Tests\\": "tests/phpunit/includes/"
},
"APIMetaModules": {
diff --git a/maintenance/CleanDuplicateScores.php 
b/maintenance/CleanDuplicateScores.php
index 994e290..7b63eec 100644
--- a/maintenance/CleanDuplicateScores.php
+++ b/maintenance/CleanDuplicateScores.php
@@ -1,6 +1,6 @@
 https://gerrit.wikimedia.org/r/403663
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia6e82b19bac1d4a7824a94a51cac1e0cb750bbde
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] integration/config[master]: Provide Java 8 for mwdumper testing

2018-01-10 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403552 )

Change subject: Provide Java 8 for mwdumper testing
..

Provide Java 8 for mwdumper testing

Change-Id: I08b3fc3b3ba063ef76e4a2e49e8615837a14b9d1
---
M jjb/mediawiki-tools.yaml
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/config 
refs/changes/52/403552/1

diff --git a/jjb/mediawiki-tools.yaml b/jjb/mediawiki-tools.yaml
index e815528..2d31181 100644
--- a/jjb/mediawiki-tools.yaml
+++ b/jjb/mediawiki-tools.yaml
@@ -65,8 +65,8 @@
 - job:
 name: 'mwdumper-maven'
 project-type: maven
-jdk: 'Debian - OpenJdk 7'
-node: ci-jessie-wikimedia
+jdk: 'Debian - OpenJdk 8'
+node: ci-stretch-wikimedia
 defaults: use-remote-zuul-shallow-clone
 concurrent: true
 triggers:

-- 
To view, visit https://gerrit.wikimedia.org/r/403552
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I08b3fc3b3ba063ef76e4a2e49e8615837a14b9d1
Gerrit-PatchSet: 1
Gerrit-Project: integration/config
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...mwdumper[master]: Light refactoring

2018-01-10 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403477 )

Change subject: Light refactoring
..

Light refactoring

Split from I3517af1cd2a2005596447d154ae338b3bf5692d9

Change-Id: I1d7a7d02623b64471721338bf33b82075010e524
---
M pom.xml
R src/org/mediawiki/dumper/filters/AfterTimeStampFilter.java
R src/org/mediawiki/dumper/filters/BeforeTimeStampFilter.java
R src/org/mediawiki/dumper/filters/ChainingDumpWriter.java
R src/org/mediawiki/dumper/filters/LatestFilter.java
R src/org/mediawiki/dumper/filters/ProgressFilter.java
R src/org/mediawiki/dumper/filters/RevisionListFilter.java
R src/org/mediawiki/dumper/filters/TimeStampFilter.java
A src/org/mediawiki/dumper/filters/predicates/BaseListFilter.java
R src/org/mediawiki/dumper/filters/predicates/ExactListFilter.java
R src/org/mediawiki/dumper/filters/predicates/ListFilter.java
R src/org/mediawiki/dumper/filters/predicates/NamespaceFilter.java
R src/org/mediawiki/dumper/filters/predicates/NotalkFilter.java
A src/org/mediawiki/dumper/filters/predicates/PredicatePageFilter.java
R src/org/mediawiki/dumper/filters/predicates/TitleMatchFilter.java
R src/org/mediawiki/dumper/input/XmlDumpReader.java
R src/org/mediawiki/dumper/wiki/Contributor.java
R src/org/mediawiki/dumper/wiki/Namespace.java
R src/org/mediawiki/dumper/wiki/NamespaceSet.java
R src/org/mediawiki/dumper/wiki/Page.java
R src/org/mediawiki/dumper/wiki/Revision.java
R src/org/mediawiki/dumper/wiki/Siteinfo.java
R src/org/mediawiki/dumper/wiki/Title.java
R src/org/mediawiki/dumper/wiki/Wikiinfo.java
R src/org/mediawiki/dumper/writers/DumpWriter.java
R src/org/mediawiki/dumper/writers/MultiWriter.java
R src/org/mediawiki/dumper/writers/sql/SqlFileStream.java
R src/org/mediawiki/dumper/writers/sql/SqlLiteral.java
R src/org/mediawiki/dumper/writers/sql/SqlServerStream.java
R src/org/mediawiki/dumper/writers/sql/SqlStream.java
R src/org/mediawiki/dumper/writers/sql/SqlWriter.java
R src/org/mediawiki/dumper/writers/sql/SqlWriter14.java
R src/org/mediawiki/dumper/writers/sql/SqlWriter15.java
R src/org/mediawiki/dumper/writers/sql/SqlWriter1_25.java
R src/org/mediawiki/dumper/writers/xml/SphinxWriter.java
R src/org/mediawiki/dumper/writers/xml/XmlDumpWriter0_10.java
R src/org/mediawiki/dumper/writers/xml/XmlDumpWriter0_3.java
R src/org/mediawiki/dumper/writers/xml/XmlWriter.java
38 files changed, 310 insertions(+), 202 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/tools/mwdumper 
refs/changes/77/403477/1

diff --git a/pom.xml b/pom.xml
index 5dc7d5c..b67cc18 100644
--- a/pom.xml
+++ b/pom.xml
@@ -15,6 +15,11 @@
 
   
 
+  com.google.guava
+  guava
+  19.0
+
+
   org.apache.commons
   commons-compress
   1.4.1
diff --git a/src/org/mediawiki/importer/AfterTimeStampFilter.java 
b/src/org/mediawiki/dumper/filters/AfterTimeStampFilter.java
similarity index 91%
rename from src/org/mediawiki/importer/AfterTimeStampFilter.java
rename to src/org/mediawiki/dumper/filters/AfterTimeStampFilter.java
index 8300fbc..ad0ffdf 100644
--- a/src/org/mediawiki/importer/AfterTimeStampFilter.java
+++ b/src/org/mediawiki/dumper/filters/AfterTimeStampFilter.java
@@ -22,14 +22,16 @@
  *
  */
 
-package org.mediawiki.importer;
+package org.mediawiki.dumper.filters;
 
 import java.io.IOException;
-import java.text.ParseException;
+
+import org.mediawiki.dumper.wiki.Revision;
+import org.mediawiki.dumper.writers.DumpWriter;
 
 public class AfterTimeStampFilter extends TimeStampFilter {
 
-   public AfterTimeStampFilter(DumpWriter sink, String timeStamp) throws 
ParseException {
+   public AfterTimeStampFilter(DumpWriter sink, String timeStamp) {
super(sink, timeStamp);
}
 
diff --git a/src/org/mediawiki/importer/BeforeTimeStampFilter.java 
b/src/org/mediawiki/dumper/filters/BeforeTimeStampFilter.java
similarity index 91%
rename from src/org/mediawiki/importer/BeforeTimeStampFilter.java
rename to src/org/mediawiki/dumper/filters/BeforeTimeStampFilter.java
index 6575c3b..7536e49 100644
--- a/src/org/mediawiki/importer/BeforeTimeStampFilter.java
+++ b/src/org/mediawiki/dumper/filters/BeforeTimeStampFilter.java
@@ -22,14 +22,16 @@
  *
  */
 
-package org.mediawiki.importer;
+package org.mediawiki.dumper.filters;
 
 import java.io.IOException;
-import java.text.ParseException;
+
+import org.mediawiki.dumper.wiki.Revision;
+import org.mediawiki.dumper.writers.DumpWriter;
 
 public class BeforeTimeStampFilter extends TimeStampFilter {
 
-   public BeforeTimeStampFilter(DumpWriter sink, String timeStamp) throws 
ParseException {
+   public BeforeTimeStampFilter(DumpWriter sink, String timeStamp) {
super(sink, timeStamp);
}
 
diff --git a/src/org/mediawiki/importer/PageFilter.java 
b/src/org/mediawiki/dumper/filters/ChainingDumpWriter.java
similarity index 72%
rename from src/org/mediawiki/importer/PageFilter.java
rename to

[MediaWiki-commits] [Gerrit] mediawiki...mwdumper[master]: Remove unused Buffer class

2018-01-10 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403476 )

Change subject: Remove unused Buffer class
..

Remove unused Buffer class

Split from I3517af1cd2a2005596447d154ae338b3bf5692d9

Change-Id: I5c381c20682c6352ddb1ebfca7493da4a6184938
---
D src/org/mediawiki/importer/Buffer.java
1 file changed, 0 insertions(+), 72 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/tools/mwdumper 
refs/changes/76/403476/1

diff --git a/src/org/mediawiki/importer/Buffer.java 
b/src/org/mediawiki/importer/Buffer.java
deleted file mode 100644
index 217ce84..000
--- a/src/org/mediawiki/importer/Buffer.java
+++ /dev/null
@@ -1,72 +0,0 @@
-/*
- * MediaWiki import/export processing tools
- * Copyright (C) 2005 by Brion Vibber
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to 
deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
- * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 
THE
- * SOFTWARE.
- *
- * $Id$
- */
-
-package org.mediawiki.importer;
-
-import java.util.IdentityHashMap;
-
-public final class Buffer {
-
-   private Buffer() {}
-
-   private static final IdentityHashMap BUFFERS = new 
IdentityHashMap();
-
-   private static Thread lastThread;
-   private static char[] lastBuffer;
-
-   public static synchronized char[] get(int capacity) {
-   final Thread thread = Thread.currentThread();
-   char[] buffer;
-
-   if (lastThread == thread) {
-   buffer = lastBuffer;
-   } else {
-   lastThread = thread;
-   buffer = lastBuffer = BUFFERS.get(thread);
-   }
-
-   if (buffer == null) {
-   buffer = lastBuffer = new char[capacity];
-   BUFFERS.put(thread, buffer);
-   } else if (buffer.length < capacity) {
-   int newsize = buffer.length * 2;
-   if (newsize < capacity)
-   newsize = capacity;
-   /*
-   // Debug!
-   System.err.println("** Growing buffer to " + newsize);
-   try {
-   throw new RuntimeException("foo");
-   } catch (RuntimeException e) {
-   e.printStackTrace();
-   }
-   */
-   buffer = lastBuffer = new char[newsize];
-   BUFFERS.put(thread, buffer);
-   }
-
-   return buffer;
-   }
-}

-- 
To view, visit https://gerrit.wikimedia.org/r/403476
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5c381c20682c6352ddb1ebfca7493da4a6184938
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/tools/mwdumper
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Cover a couple of conditions

2018-01-10 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403458 )

Change subject: Cover a couple of conditions
..

Cover a couple of conditions

Change-Id: I40bedcee1c6da4d837ab9b9109ad6d26e0164f79
---
M tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php
1 file changed, 31 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/58/403458/1

diff --git a/tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php 
b/tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php
index b55bd01..dd47f50 100644
--- a/tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php
+++ b/tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php
@@ -51,6 +51,7 @@
$this->user->setOption( 'rcOresDamagingPref', 'maybebad' );
$this->user->setOption( 'oresHighlight', 1 );
$this->user->setOption( 'ores-damaging-flag-rc', 1 );
+   $this->user->setOption( 'oresRCHideNonDamaging', 1 );
$this->user->setOption( 'rcenhancedfilters-disable', true );
$this->user->saveSettings();
 
@@ -404,6 +405,36 @@
$maybebadFilter = $damagingFilterGroup->getFilter( 'maybebad' );
$this->assertNotNull( $maybebadFilter );
 
+   $this->assertEquals( 'maybebad', 
$damagingFilterGroup->getDefault() );
+
+   $goodfaithFilterGroup = 
$changesListSpecialPage->getFilterGroup( 'goodfaith' );
+   $this->assertNull( $goodfaithFilterGroup );
+   }
+
+   /**
+* @covers 
ORES\Hooks\ChangesListHooksHandler::onChangesListSpecialPageStructuredFilters
+*/
+   public function 
testOnChangesListSpecialPageStructuredFilters_Watchlist() {
+   $this->user->setOption( 'oresWatchlistHideNonDamaging', 0 );
+   $this->user->setOption( 'oresHighlight', 1 );
+
+   $changesListSpecialPage = new \SpecialWatchlist();
+   $changesListSpecialPage->setContext( $this->context );
+   $wrappedClsp = TestingAccessWrapper::newFromObject( 
$changesListSpecialPage );
+   $wrappedClsp->registerFilters();
+
+   
ChangesListHooksHandler::onChangesListSpecialPageStructuredFilters( 
$changesListSpecialPage );
+
+   $damagingFilterGroup = $changesListSpecialPage->getFilterGroup( 
'damaging' );
+   $this->assertNotNull( $damagingFilterGroup );
+   $maybebadFilter = $damagingFilterGroup->getFilter( 'maybebad' );
+   $this->assertNotNull( $maybebadFilter );
+
+   $this->assertEquals( '', $damagingFilterGroup->getDefault() );
+
+   $filterJsData = $damagingFilterGroup->getFilter( 'likelybad' 
)->getJsData();
+   $this->assertEquals( 'c4', 
$filterJsData['defaultHighlightColor'] );
+
$goodfaithFilterGroup = 
$changesListSpecialPage->getFilterGroup( 'goodfaith' );
$this->assertNull( $goodfaithFilterGroup );
}

-- 
To view, visit https://gerrit.wikimedia.org/r/403458
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I40bedcee1c6da4d837ab9b9109ad6d26e0164f79
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Move some sharable methods into a test helper

2018-01-10 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403450 )

Change subject: Move some sharable methods into a test helper
..

Move some sharable methods into a test helper

Change-Id: I98b57fdf743cd31048de436a42d8f37c7c1c8679
---
M extension.json
M tests/phpunit/includes/Api/ApiIntegrationTest.php
A tests/phpunit/includes/TestHelper.php
3 files changed, 83 insertions(+), 61 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/50/403450/1

diff --git a/extension.json b/extension.json
index 9745ffb..d7f8e5d 100644
--- a/extension.json
+++ b/extension.json
@@ -12,7 +12,8 @@
"MediaWiki": ">= 1.31.0"
},
"AutoloadNamespaces": {
-   "ORES\\": "includes/"
+   "ORES\\": "includes/",
+   "ORES\\Tests\\": "tests/phpunit/includes/"
},
"APIMetaModules": {
"ores": "ORES\\ApiQueryORES"
diff --git a/tests/phpunit/includes/Api/ApiIntegrationTest.php 
b/tests/phpunit/includes/Api/ApiIntegrationTest.php
index 14e605d..a1d1b65 100644
--- a/tests/phpunit/includes/Api/ApiIntegrationTest.php
+++ b/tests/phpunit/includes/Api/ApiIntegrationTest.php
@@ -2,16 +2,10 @@
 
 namespace ORES\Tests\Api;
 
-use ContentHandler;
-use MediaWiki\Linker\LinkTarget;
-use MediaWiki\MediaWikiServices;
 use ORES\Storage\HashModelLookup;
-use ORES\Storage\ModelLookup;
-use Revision;
-use Title;
 use TitleValue;
-use User;
-use WikiPage;
+
+use ORES\Tests\TestHelper;
 
 /**
  * @group API
@@ -26,10 +20,7 @@
public function __construct( $name = null, array $data = [], $dataName 
= '' ) {
parent::__construct( $name, $data, $dataName );
 
-   $this->tablesUsed[] = 'recentchanges';
-   $this->tablesUsed[] = 'page';
-   $this->tablesUsed[] = 'ores_model';
-   $this->tablesUsed[] = 'ores_classification';
+   $this->tablesUsed = TestHelper::getTablesUsed();
}
 
protected function setUp() {
@@ -37,9 +28,8 @@
 
self::$users['ORESApiIntegrationTestUser'] = 
$this->getMutableTestUser();
$this->doLogin( 'ORESApiIntegrationTestUser' );
-   wfGetDB( DB_MASTER )->delete( 'recentchanges', '*', __METHOD__ 
);
-   wfGetDB( DB_MASTER )->delete( 'ores_model', '*', __METHOD__ );
-   wfGetDB( DB_MASTER )->delete( 'ores_classification', '*', 
__METHOD__ );
+
+   TestHelper::clearOresTables();
 
$this->setMwGlobals(
[
@@ -112,8 +102,8 @@
 
public function testListRecentChanges_getOresScores() {
$target = new TitleValue( 0, 'ORESApiIntegrationTestPage' );
-   $status = $this->doPageEdit( $this->getLoggedInTestUser(), 
$target, 'Create the page' );
-   $this->insertOresData(
+   $status = TestHelper::doPageEdit( $this->getLoggedInTestUser(), 
$target, 'Create the page' );
+   TestHelper::insertOresData(
$status->getValue()['revision'],
[ 'damaging' => 0.4, 'goodfaith' => 0.7 ]
);
@@ -131,43 +121,8 @@
$this->assertEquals( 
$result[0]['query']['recentchanges'][0]['oresscores'], $expected );
}
 
-   private function doPageEdit( User $user, LinkTarget $target, $summary ) 
{
-   static $i = 0;
-
-   $title = Title::newFromLinkTarget( $target );
-   $page = WikiPage::factory( $title );
-   $status = $page->doEditContent(
-   ContentHandler::makeContent( __CLASS__ . $i++, $title ),
-   $summary,
-   0,
-   false,
-   $user
-   );
-   if ( !$status->isOK() ) {
-   $this->fail();
-   }
-   return $status;
-   }
-
private function getLoggedInTestUser() {
return self::$users['ORESApiIntegrationTestUser']->getUser();
-   }
-
-   private function insertOresData( Revision $revision, $scores ) {
-   /** @var ModelLookup $modelLookup */
-   $modelLookup = MediaWikiServices::getInstance()->getService( 
'ORESModelLookup' );
-   // TODO: Use ScoreStorage
-   $dbData = [];
-   foreach ( $scores as $modelName => $score ) {
-   $dbData[] = [
-   'oresc_rev' => $revision->getId(),
-   'oresc_model' => $modelLookup->getModelId( 
$modelName ),
-   'oresc_class' => 1,
-   'oresc_probability' => $score,
-   'oresc_is_predicted' => 0
-   ];
-   }
-   wfGetDB( DB_MASTER )->insert(

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: [WIP] Add more WatchedItemQueryServiceExtension tests.

2018-01-09 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403234 )

Change subject: [WIP] Add more WatchedItemQueryServiceExtension tests.
..

[WIP] Add more WatchedItemQueryServiceExtension tests.

Bug: T184140
Change-Id: I27225a9a0e67fc6bc8024f5e7d4b6c8c29a730c3
---
M tests/phpunit/includes/Api/ApiIntegrationTest.php
A tests/phpunit/includes/TestHelper.php
M tests/phpunit/includes/WatchedItemQueryServiceExtensionTest.php
3 files changed, 87 insertions(+), 29 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/34/403234/1

diff --git a/tests/phpunit/includes/Api/ApiIntegrationTest.php 
b/tests/phpunit/includes/Api/ApiIntegrationTest.php
index 14e605d..31906f2 100644
--- a/tests/phpunit/includes/Api/ApiIntegrationTest.php
+++ b/tests/phpunit/includes/Api/ApiIntegrationTest.php
@@ -13,6 +13,8 @@
 use User;
 use WikiPage;
 
+use ORES\Tests\TestHelper;
+
 /**
  * @group API
  * @group Database
@@ -26,10 +28,7 @@
public function __construct( $name = null, array $data = [], $dataName 
= '' ) {
parent::__construct( $name, $data, $dataName );
 
-   $this->tablesUsed[] = 'recentchanges';
-   $this->tablesUsed[] = 'page';
-   $this->tablesUsed[] = 'ores_model';
-   $this->tablesUsed[] = 'ores_classification';
+   $this->tablesUsed = TestHelper::getTablesUsed();
}
 
protected function setUp() {
@@ -37,9 +36,8 @@
 
self::$users['ORESApiIntegrationTestUser'] = 
$this->getMutableTestUser();
$this->doLogin( 'ORESApiIntegrationTestUser' );
-   wfGetDB( DB_MASTER )->delete( 'recentchanges', '*', __METHOD__ 
);
-   wfGetDB( DB_MASTER )->delete( 'ores_model', '*', __METHOD__ );
-   wfGetDB( DB_MASTER )->delete( 'ores_classification', '*', 
__METHOD__ );
+
+   TestHelper::clearOresTables();
 
$this->setMwGlobals(
[
@@ -113,7 +111,7 @@
public function testListRecentChanges_getOresScores() {
$target = new TitleValue( 0, 'ORESApiIntegrationTestPage' );
$status = $this->doPageEdit( $this->getLoggedInTestUser(), 
$target, 'Create the page' );
-   $this->insertOresData(
+   TestHelper::insertOresData(
$status->getValue()['revision'],
[ 'damaging' => 0.4, 'goodfaith' => 0.7 ]
);
@@ -153,23 +151,6 @@
return self::$users['ORESApiIntegrationTestUser']->getUser();
}
 
-   private function insertOresData( Revision $revision, $scores ) {
-   /** @var ModelLookup $modelLookup */
-   $modelLookup = MediaWikiServices::getInstance()->getService( 
'ORESModelLookup' );
-   // TODO: Use ScoreStorage
-   $dbData = [];
-   foreach ( $scores as $modelName => $score ) {
-   $dbData[] = [
-   'oresc_rev' => $revision->getId(),
-   'oresc_model' => $modelLookup->getModelId( 
$modelName ),
-   'oresc_class' => 1,
-   'oresc_probability' => $score,
-   'oresc_is_predicted' => 0
-   ];
-   }
-   wfGetDB( DB_MASTER )->insert( 'ores_classification', $dbData );
-   }
-
private function doListRecentChangesRequest( array $params = [] ) {
return $this->doApiRequest(
array_merge(
@@ -184,7 +165,7 @@
public function testListRecentChanges_showOresReview() {
$target = new TitleValue( 0, 'ORESApiIntegrationTestPage' );
$status = $this->doPageEdit( $this->getLoggedInTestUser(), 
$target, 'Create the page' );
-   $this->insertOresData(
+   TestHelper::insertOresData(
$status->getValue()['revision'],
[ 'damaging' => 0.6, 'goodfaith' => 0.3 ]
);
@@ -205,7 +186,7 @@
public function testListRecentChanges_showOresReviewNotNeedingReview() {
$target = new TitleValue( 0, 'ORESApiIntegrationTestPage' );
$status = $this->doPageEdit( $this->getLoggedInTestUser(), 
$target, 'Create the page' );
-   $this->insertOresData(
+   TestHelper::insertOresData(
$status->getValue()['revision'],
[ 'damaging' => 0.4, 'goodfaith' => 0.7 ]
);
@@ -220,7 +201,7 @@
public function testListRecentChanges_showNotOresReview() {
$target = new TitleValue( 0, 'ORESApiIntegrationTestPage' );
$status = $this->doPageEdit( $this->getLoggedInTestUser(), 
$target, 'Create the page' );
-

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Tests for WatchedItemQueryServiceExtension

2018-01-09 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403227 )

Change subject: Tests for WatchedItemQueryServiceExtension
..

Tests for WatchedItemQueryServiceExtension

Bug: T184140
Change-Id: I074762b884cd34ae13f2fd71d654f6ddd309c8b3
---
A tests/phpunit/includes/WatchedItemQueryServiceExtensionTest.php
1 file changed, 119 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/27/403227/1

diff --git a/tests/phpunit/includes/WatchedItemQueryServiceExtensionTest.php 
b/tests/phpunit/includes/WatchedItemQueryServiceExtensionTest.php
new file mode 100644
index 000..11523e3
--- /dev/null
+++ b/tests/phpunit/includes/WatchedItemQueryServiceExtensionTest.php
@@ -0,0 +1,119 @@
+setMwGlobals( [
+   'wgOresFiltersThresholds' => [
+   'damaging' => [
+   'maybebad' => [ 'min' => 0.16, 'max' => 
1 ],
+   'likelybad' => [ 'min' => 0.56, 'max' 
=> 1 ],
+   ]
+   ],
+   'wgOresWikiId' => 'testwiki',
+   ] );
+
+   $this->user = static::getTestUser()->getUser();
+   $this->user->setOption( 'ores-enabled', 1 );
+   $this->user->setOption( 'oresDamagingPref', 'maybebad' );
+   $this->user->setOption( 'oresHighlight', 1 );
+   $this->user->setOption( 'ores-damaging-flag-rc', 1 );
+   $this->user->saveSettings();
+
+   $modelData = [ 'damaging' => [ 'id' => 5, 'version' => '0.0.2' 
] ];
+   $this->setService( 'ORESModelLookup', new HashModelLookup( 
$modelData ) );
+   }
+
+   /**
+* @covers 
ORES\WatchedItemQueryServiceExtensionTest::modifyWatchedItemsWithRCInfoQuery
+*/
+   public function testModifyWatchedItemsWithRCInfoQuery_review() {
+   $options = [
+   'filters' => [ 'oresreview' ],
+   'includeFields' => [ 'oresscores' ],
+   'usedInGenerator' => false,
+   ];
+   $tables = [];
+   $fields = [];
+   $conds = [];
+   $dbOptions = [];
+   $joinConds = [];
+   $db = wfGetDB( DB_MASTER );
+   $service = new WatchedItemQueryServiceExtension();
+   $service->modifyWatchedItemsWithRCInfoQuery(
+   $this->user, $options, $db, $tables, $fields, $conds,
+   $dbOptions, $joinConds );
+
+   $this->assertEquals( [
+   'rc_this_oldid',
+   'rc_type',
+   ], $fields );
+   $this->assertEquals( [
+   'ores_model',
+   'ores_classification',
+   ], $tables );
+   $this->assertEquals( [
+   'oresc_probability > \'0.16\'',
+   ], $conds );
+   $this->assertEquals( [
+   'ores_classification' => [ 'INNER JOIN', [
+   'rc_this_oldid=oresc_rev',
+   'oresc_model' => 5,
+   'oresc_class' => 1,
+   ], ],
+   ], $joinConds );
+   }
+
+   /**
+* @covers 
ORES\WatchedItemQueryServiceExtensionTest::modifyWatchedItemsWithRCInfoQuery
+*/
+   public function testModifyWatchedItemsWithRCInfoQuery_not_review() {
+   $options = [
+   'filters' => [ '!oresreview' ],
+   'includeFields' => [ 'oresscores' ],
+   'usedInGenerator' => false,
+   ];
+   $tables = [];
+   $fields = [];
+   $conds = [];
+   $dbOptions = [];
+   $joinConds = [];
+   $db = wfGetDB( DB_MASTER );
+   $service = new WatchedItemQueryServiceExtension();
+   $service->modifyWatchedItemsWithRCInfoQuery(
+   $this->user, $options, $db, $tables, $fields, $conds,
+   $dbOptions, $joinConds );
+
+   $this->assertEquals( [
+   'rc_this_oldid',
+   'rc_type',
+   ], $fields );
+   $this->assertEquals( [
+   'ores_model',
+   'ores_classification',
+   ], $tables );
+   $this->assertEquals( [
+   '(oresc_probability <= \'0.16\') OR (oresc_probability 
IS NULL)',
+   ], $conds );
+   $this->assertEquals( [
+   'ores_classification' => [ 'LEFT JOIN', [
+   'rc_this_oldid=oresc_rev',
+

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Cover some of Hooks

2018-01-08 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/403068 )

Change subject: Cover some of Hooks
..

Cover some of Hooks

Bug: T184140
Change-Id: I49fc3c3601a91758ca7d44c5f4415bfce7c71903
---
M tests/phpunit/includes/HooksTest.php
1 file changed, 44 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/68/403068/1

diff --git a/tests/phpunit/includes/HooksTest.php 
b/tests/phpunit/includes/HooksTest.php
index b770b9b..c81515d 100644
--- a/tests/phpunit/includes/HooksTest.php
+++ b/tests/phpunit/includes/HooksTest.php
@@ -3,9 +3,12 @@
 namespace ORES\Tests;
 
 use IContextSource;
+use JobQueueGroup;
 use ORES\Hooks;
 use ORES\Hooks\PreferencesHookHandler;
 use ORES\Storage\HashModelLookup;
+use ORES\Storage\ScoreStorage;
+use RecentChange;
 use RequestContext;
 use SpecialPage;
 use User;
@@ -44,6 +47,47 @@
}
 
/**
+* @covers ORES\Hooks::onRecentChange_save
+*/
+   public function testOnRecentChange_save() {
+   JobQueueGroup::singleton()->get( 'ORESFetchScoreJob' 
)->delete();
+
+   $rc = RecentChange::newFromRow( (object)[
+   'rc_patrolled' => 0,
+   'rc_timestamp' => '20150921134808',
+   'rc_deleted' => 0,
+   'rc_comment' => '',
+   'rc_comment_text' => '',
+   'rc_comment_data' => null,
+   'rc_type' => RC_EDIT,
+   'rc_this_oldid' => mt_rand(1000, ),
+   ] );
+   Hooks::onRecentChange_save( $rc );
+
+   $this->assertFalse( JobQueueGroup::singleton()->get( 
'ORESFetchScoreJob' )->isEmpty() );
+   }
+
+   /**
+* @covers ORES\Hooks::onRecentChangesPurgeRows
+*/
+   public function testOnRecentChangesPurgeRows() {
+   $revIds = [ 1, 5, 8, 13 ];
+   $rows = array_map( function( $id ) {
+   return (object)[ 'rc_this_oldid' => $id ];
+   }, $revIds );
+
+   $mock = $this->getMockBuilder( ScoreStorage::class )
+   ->getMock();
+   $mock->expects( $this->once() )
+   ->method( 'purgeRows' )
+   ->with( $this->equalTo( $revIds ) );
+
+   $this->setService( 'ORESScoreStorage', $mock );
+
+   Hooks::onRecentChangesPurgeRows( $rows );
+   }
+
+   /**
 * @covers ORES\Hooks\PreferencesHookHandler::onGetPreferences
 * @todo Move to a dedicated file
 */

-- 
To view, visit https://gerrit.wikimedia.org/r/403068
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I49fc3c3601a91758ca7d44c5f4415bfce7c71903
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Mostly cover ChangesListHooksHandler

2018-01-08 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/402878 )

Change subject: Mostly cover ChangesListHooksHandler
..

Mostly cover ChangesListHooksHandler

Bug: T184140
Change-Id: I306812039bca488ea037a0e8bcfe7b441d7179e6
---
M tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php
1 file changed, 87 insertions(+), 49 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/78/402878/1

diff --git a/tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php 
b/tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php
index 1031cc5..31d8129 100644
--- a/tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php
+++ b/tests/phpunit/includes/Hooks/ChangesListHooksHandlerTest.php
@@ -15,6 +15,7 @@
 use RequestContext;
 use SpecialPage;
 use User;
+use Wikimedia\TestingAccessWrapper;
 
 /**
  * @group ORES
@@ -56,13 +57,14 @@
$this->context = self::getContext( $this->user );
}
 
-   /**
-* @covers ORES\Hooks\ChangesListHooksHandler::getScoreRecentChangesList
-*/
-   public function testOresRCObj() {
+   protected function makeRcEntry($isDamaging = false) {
$row = new \stdClass();
$row->ores_damaging_threshold = 0.2;
-   $row->ores_damaging_score = 0.3;
+   if ( $isDamaging ) {
+   $row->ores_damaging_score = 0.3;
+   } else {
+   $row->ores_damaging_score = 0.1;
+   }
$row->rc_patrolled = 0;
$row->rc_timestamp = '20150921134808';
$row->rc_deleted = 0;
@@ -71,11 +73,18 @@
$row->rc_comment_data = null;
 
$rc = RecentChange::newFromRow( $row );
-   $this->assertTrue( 
ChangesListHooksHandler::getScoreRecentChangesList( $rc, $this->context ) );
+   return $rc;
+   }
 
-   $row->ores_damaging_threshold = 0.4;
-   $rc = RecentChange::newFromRow( $row );
-   $this->assertFalse( 
ChangesListHooksHandler::getScoreRecentChangesList( $rc, $this->context ) );
+   /**
+* @covers ORES\Hooks\ChangesListHooksHandler::getScoreRecentChangesList
+*/
+   public function testGetScoreRecentChangesList() {
+   $this->assertTrue( 
ChangesListHooksHandler::getScoreRecentChangesList(
+   $this->makeRcEntry( true ), $this->context ) );
+
+   $this->assertFalse( 
ChangesListHooksHandler::getScoreRecentChangesList(
+   $this->makeRcEntry( false ), $this->context ) );
}
 
/**
@@ -193,16 +202,7 @@
 * @covers 
ORES\Hooks\ChangesListHooksHandler::onEnhancedChangesListModifyLineData
 */
public function testOnEnhancedChangesListModifyLineDataDamaging() {
-   $row = new \stdClass();
-   $row->ores_damaging_threshold = 0.2;
-   $row->ores_damaging_score = 0.3;
-   $row->rc_patrolled = 0;
-   $row->rc_timestamp = '20150921134808';
-   $row->rc_deleted = 0;
-   $row->rc_comment = '';
-   $row->rc_comment_text = '';
-   $row->rc_comment_data = null;
-   $rc = RecentChange::newFromRow( $row );
+   $rc = $this->makeRcEntry( true );
$rc = RCCacheEntry::newFromParent( $rc );
 
$ecl = $this->getMockBuilder( EnhancedChangesList::class )
@@ -238,16 +238,7 @@
 * @covers 
ORES\Hooks\ChangesListHooksHandler::onEnhancedChangesListModifyLineData
 */
public function testOnEnhancedChangesListModifyLineDataNonDamaging() {
-   $row = new \stdClass();
-   $row->ores_damaging_threshold = 0.4;
-   $row->ores_damaging_score = 0.3;
-   $row->rc_patrolled = 0;
-   $row->rc_timestamp = '20150921134808';
-   $row->rc_deleted = 0;
-   $row->rc_comment = '';
-   $row->rc_comment_text = '';
-   $row->rc_comment_data = null;
-   $rc = RecentChange::newFromRow( $row );
+   $rc = $this->makeRcEntry( false );
$rc = RCCacheEntry::newFromParent( $rc );
 
$ecl = $this->getMockBuilder( EnhancedChangesList::class )
@@ -287,16 +278,7 @@
 * @covers 
ORES\Hooks\ChangesListHooksHandler::onOldChangesListRecentChangesLine
 */
public function testOnOldChangesListModifyLineDataDamaging() {
-   $row = new \stdClass();
-   $row->ores_damaging_threshold = 0.2;
-   $row->ores_damaging_score = 0.3;
-   $row->rc_patrolled = 0;
-   $row->rc_timestamp = '20150921134808';
-   $row->rc_deleted = 0;
-   $row->rc_comment = '';
-   $row->rc_comment_text = '';

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Add an "unless" condition to npm downgrade

2018-01-04 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/402163 )

Change subject: Add an "unless" condition to npm downgrade
..

Add an "unless" condition to npm downgrade

This was running on every provision.

Change-Id: Iaedae6fe7c51e6cc02383a4e78388498f7c446d9
---
M puppet/modules/npm/manifests/init.pp
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/63/402163/1

diff --git a/puppet/modules/npm/manifests/init.pp 
b/puppet/modules/npm/manifests/init.pp
index dde3fdd..85dc77b 100644
--- a/puppet/modules/npm/manifests/init.pp
+++ b/puppet/modules/npm/manifests/init.pp
@@ -46,6 +46,7 @@
 # Although the ticket is closed, the issue is still present, so downgrade 
to npm 2
 exec { 'downgrade_npm':
   command => '/usr/bin/npm install -g npm@latest-2',
+  unless  => '/usr/bin/npm --version | grep -qE "^2[.]"',
   user=> 'root',
   require => [
   Package['nodejs'],

-- 
To view, visit https://gerrit.wikimedia.org/r/402163
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iaedae6fe7c51e6cc02383a4e78388498f7c446d9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Add ordering between script and its execution

2018-01-04 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/402110 )

Change subject: Add ordering between script and its execution
..

Add ordering between script and its execution

Bug: T183605
Change-Id: Iab45464609aa4e763d0311d28b8314f991b04375
---
M puppet/modules/wikilabels/manifests/init.pp
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/10/402110/1

diff --git a/puppet/modules/wikilabels/manifests/init.pp 
b/puppet/modules/wikilabels/manifests/init.pp
index e00556d..76a646c 100644
--- a/puppet/modules/wikilabels/manifests/init.pp
+++ b/puppet/modules/wikilabels/manifests/init.pp
@@ -67,6 +67,7 @@
 unless  => "psql -lqt | cut -d \\| -f 1 | grep -qw ${db_name}",
 user=> 'postgres',
 require => [
+  File[$db_script],
   Package['postgresql'],
   VirtualEnv::Package['wikilabels'],
 ],

-- 
To view, visit https://gerrit.wikimedia.org/r/402110
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iab45464609aa4e763d0311d28b8314f991b04375
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[stretch-migration]: Add ordering between script and its execution

2018-01-03 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401805 )

Change subject: Add ordering between script and its execution
..

Add ordering between script and its execution

Bug: T183605
Change-Id: Iab45464609aa4e763d0311d28b8314f991b04375
---
M puppet/modules/wikilabels/manifests/init.pp
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/05/401805/1

diff --git a/puppet/modules/wikilabels/manifests/init.pp 
b/puppet/modules/wikilabels/manifests/init.pp
index 6cc0379..7a597d8 100644
--- a/puppet/modules/wikilabels/manifests/init.pp
+++ b/puppet/modules/wikilabels/manifests/init.pp
@@ -67,6 +67,7 @@
 unless  => "psql -lqt | cut -d \\| -f 1 | grep -qw ${db_name}",
 user=> 'postgres',
 require => [
+  File[$db_script],
   Package['postgresql'],
   VirtualEnv::Package['wikilabels'],
 ],

-- 
To view, visit https://gerrit.wikimedia.org/r/401805
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iab45464609aa4e763d0311d28b8314f991b04375
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: stretch-migration
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[stretch-migration]: Use absolute scope for systemd template variables

2018-01-03 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401791 )

Change subject: Use absolute scope for systemd template variables
..

Use absolute scope for systemd template variables

Bug: T184077
Change-Id: I123ba4d1b19c4e5c2c51f9b042e3ac06ffae3362
---
M puppet/modules/ores/templates/systemd/ores-celery.erb
M puppet/modules/ores/templates/systemd/ores-wsgi.erb
M puppet/modules/wikilabels/manifests/init.pp
M puppet/modules/wikilabels/templates/systemd/wikilabels.erb
4 files changed, 7 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/91/401791/1

diff --git a/puppet/modules/ores/templates/systemd/ores-celery.erb 
b/puppet/modules/ores/templates/systemd/ores-celery.erb
index 21493e8..bc0c196 100644
--- a/puppet/modules/ores/templates/systemd/ores-celery.erb
+++ b/puppet/modules/ores/templates/systemd/ores-celery.erb
@@ -11,8 +11,8 @@
 User=www-data
 Group=www-data
 SyslogIdentifier=ores-celery
-WorkingDirectory=<%= @ores_root %>
-ExecStart=<%= @venv_dir %>/bin/ores applications.celery --logging-config <%= 
@logging_config %>
+WorkingDirectory=<%= scope['::ores::ores_root'] %>
+ExecStart=<%= scope['::ores::venv_dir'] %>/bin/ores applications.celery 
--logging-config <%= scope['::ores::logging_config'] %>
 Restart=always
 
 [Install]
diff --git a/puppet/modules/ores/templates/systemd/ores-wsgi.erb 
b/puppet/modules/ores/templates/systemd/ores-wsgi.erb
index afeb81b..97bfab6 100644
--- a/puppet/modules/ores/templates/systemd/ores-wsgi.erb
+++ b/puppet/modules/ores/templates/systemd/ores-wsgi.erb
@@ -11,8 +11,8 @@
 User=www-data
 Group=www-data
 SyslogIdentifier=ores-wsgi
-WorkingDirectory=<%= @ores_root %>
-ExecStart=<%= @venv_dir %>/bin/ores applications.wsgi --port=<%= @port %> 
--processes=1 --logging-config <%= @logging_config %>
+WorkingDirectory=<%= scope['::ores::ores_root'] %>
+ExecStart=<%= scope['::ores::venv_dir'] %>/bin/ores applications.wsgi 
--port=<%= scope['::ores::port'] %> --processes=1 --logging-config <%= 
scope['::ores::logging_config'] %>
 Restart=always
 
 [Install]
diff --git a/puppet/modules/wikilabels/manifests/init.pp 
b/puppet/modules/wikilabels/manifests/init.pp
index e00556d..6cc0379 100644
--- a/puppet/modules/wikilabels/manifests/init.pp
+++ b/puppet/modules/wikilabels/manifests/init.pp
@@ -76,7 +76,7 @@
 command => "echo y | sudo -u www-data ${deploy_dir}/bin/wikilabels 
load_schema --reload-test-data",
 unless  => "sudo -u postgres psql -d ${db_name} -c \"SELECT 
'campaign'::regclass\"",
 cwd => $repo_dir,
-require =>[
+require => [
   Exec['create wikilabels database'],
   Virtualenv::Package['wikilabels'],
 ],
diff --git a/puppet/modules/wikilabels/templates/systemd/wikilabels.erb 
b/puppet/modules/wikilabels/templates/systemd/wikilabels.erb
index e5c651d..8b3fdeb 100644
--- a/puppet/modules/wikilabels/templates/systemd/wikilabels.erb
+++ b/puppet/modules/wikilabels/templates/systemd/wikilabels.erb
@@ -11,8 +11,8 @@
 User=www-data
 Group=www-data
 SyslogIdentifier=wikilabels
-WorkingDirectory=<%= @repo_dir %>
-ExecStart=<%= @deploy_dir %>/bin/wikilabels dev_server --port=<%= @port %>
+WorkingDirectory=<%= scope['::wikilabels::repo_dir'] %>
+ExecStart=<%= scope['::wikilabels::deploy_dir'] %>/bin/wikilabels dev_server 
--port=<%= scope['::wikilabels::port'] %>
 Restart=always
 
 [Install]

-- 
To view, visit https://gerrit.wikimedia.org/r/401791
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I123ba4d1b19c4e5c2c51f9b042e3ac06ffae3362
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: stretch-migration
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[stretch-migration]: Check out both ores and revscoring as editable modules

2018-01-03 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401780 )

Change subject: Check out both ores and revscoring as editable modules
..

Check out both ores and revscoring as editable modules

Also, install ores[redis] dependencies in a way that hides the
details from puppet.

It seems that we no longer need the manual installation of revscoring
dependencies.  This went away as mysteriously as it began.

Bug: T181850
Change-Id: Ifcaa471006a90b8e76118fbb4e91523656ae740d
---
M puppet/modules/ores/manifests/init.pp
M puppet/modules/ores/templates/systemd/ores-celery.erb
M puppet/modules/ores/templates/systemd/ores-wsgi.erb
M puppet/modules/role/manifests/ores.pp
4 files changed, 50 insertions(+), 24 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/80/401780/1

diff --git a/puppet/modules/ores/manifests/init.pp 
b/puppet/modules/ores/manifests/init.pp
index 089d08d..0e2c053 100644
--- a/puppet/modules/ores/manifests/init.pp
+++ b/puppet/modules/ores/manifests/init.pp
@@ -26,47 +26,75 @@
 # revscoring
 require_package('python3-dev', 'g++', 'gfortran', 'liblapack-dev', 
'libopenblas-dev', 'libenchant-dev')
 
+file { $deploy_dir:
+ensure => directory,
+}
+
 # ORES (in a venv as it needs Python 3)
-virtualenv::environment { $deploy_dir:
+$venv_dir = "${deploy_dir}/venv"
+virtualenv::environment { $venv_dir:
 ensure  => present,
 owner   => $::share_owner,
 group   => $::share_group,
 python  => 'python3',
-require => Package['python3-dev', 'g++', 'gfortran', 'liblapack-dev', 
'libopenblas-dev', 'libenchant-dev'],
+require => [
+Package['python3-dev', 'g++', 'gfortran', 'liblapack-dev', 
'libopenblas-dev', 'libenchant-dev'],
+File[$deploy_dir],
+],
 }
-virtualenv::package { 'ores':
-package  => 'git+https://github.com/wiki-ai/ores.git#egg=ores',
-path => $deploy_dir,
+
+$sources_dir = "${deploy_dir}/src"
+file { $sources_dir:
+ensure => directory,
+}
+git::clone { 'revscoring':
+directory => "${sources_dir}/revscoring",
+remote=> 'https://github.com/wiki-ai/revscoring',
+require   => File[$sources_dir],
+}
+$ores_root = "${sources_dir}/ores"
+git::clone { 'ores':
+directory => $ores_root,
+remote=> 'https://github.com/wiki-ai/ores',
+require   => File[$sources_dir],
+}
+
+virtualenv::package { 'revscoring':
+package  => "${sources_dir}/revscoring",
+path => $venv_dir,
 editable => true,
+require  => Git::Clone['revscoring'],
 }
-#FIXME this should happen as part of normal dependency management but for 
some reason it doesn't
-# pylru probably needs to be fixed in the revscoring pakcage, redis in ores
-exec { 'pip_install_revscoring_dependencies_hack':
-command => "curl 
https://raw.githubusercontent.com/wiki-ai/revscoring/master/requirements.txt | 
${deploy_dir}/bin/pip install pylru redis -r /dev/stdin",
-cwd => $deploy_dir,
-subscribe   => Virtualenv::Package['ores'],
-refreshonly => true,
+
+virtualenv::package { 'ores':
+# Tricky syntax to get pip to install ores with the extra "redis"
+# dependencies specified in setup.py
+package  => "${ores_root}[redis]",
+path => $venv_dir,
+editable => true,
+require  => [
+Virtualenv::Package['revscoring'],
+Git::Clone['ores'],
+],
 }
-$repo_dir = "${deploy_dir}/src/ores"
 
 apache::reverse_proxy { 'ores':
 port => $port,
 }
 
-$cfg_file = "${repo_dir}/config/999-vagrant.yaml"
+$cfg_file = "${ores_root}/config/999-vagrant.yaml"
 file { $cfg_file:
 ensure  => present,
 content => template('ores/ores.yaml.erb'),
-require => Virtualenv::Package['ores'],
+require => Git::Clone['ores'],
 }
 
 systemd::service { 'ores-wsgi':
 ensure => present,
 service_params => {
 require   => [
-VirtualEnv::Package['ores'],
+Virtualenv::Package['ores'],
 Class['mediawiki::ready_service'],
-Exec['pip_install_revscoring_dependencies_hack'],
 Apache::Site['ores'],
 ],
 subscribe => [
@@ -78,9 +106,8 @@
 ensure => present,
 service_params => {
 require   => [
-VirtualEnv::Package['ores'],
+Virtualenv::Package['ores'],
 Class['mediawiki::ready_service'],
-Exec['pip_install_revscoring_dependencies_hack'],
 Apache::Site['ores'],
 ],
 subscribe => [
diff --git

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[stretch-migration]: Configure ORES logging

2018-01-03 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401781 )

Change subject: Configure ORES logging
..

Configure ORES logging

Bug: T181850
Bug: T182614
Change-Id: I5042b25a17accbfb5ec430d4c3a7b92ea45336fb
---
M puppet/modules/ores/manifests/init.pp
A puppet/modules/ores/templates/logging.yaml.erb
M puppet/modules/ores/templates/systemd/ores-celery.erb
M puppet/modules/ores/templates/systemd/ores-wsgi.erb
4 files changed, 54 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/81/401781/1

diff --git a/puppet/modules/ores/manifests/init.pp 
b/puppet/modules/ores/manifests/init.pp
index 0e2c053..7412885 100644
--- a/puppet/modules/ores/manifests/init.pp
+++ b/puppet/modules/ores/manifests/init.pp
@@ -89,6 +89,13 @@
 require => Git::Clone['ores'],
 }
 
+$logging_config = "${ores_root}/logging_config.yaml"
+file { $logging_config:
+ensure  => present,
+content => template('ores/logging.yaml.erb'),
+require => Virtualenv::Package['ores'],
+}
+
 systemd::service { 'ores-wsgi':
 ensure => present,
 service_params => {
@@ -99,6 +106,7 @@
 ],
 subscribe => [
 File[$cfg_file],
+File[$logging_config],
 ],
 },
 }
@@ -112,6 +120,7 @@
 ],
 subscribe => [
 File[$cfg_file],
+File[$logging_config],
 ],
 },
 }
diff --git a/puppet/modules/ores/templates/logging.yaml.erb 
b/puppet/modules/ores/templates/logging.yaml.erb
new file mode 100644
index 000..96c4642
--- /dev/null
+++ b/puppet/modules/ores/templates/logging.yaml.erb
@@ -0,0 +1,43 @@
+version: 1
+
+root:
+  handlers: [file]
+
+loggers:
+
+  revscoring:
+level: DEBUG
+  ores:
+level: DEBUG
+  ores.metrics_collectors.logger:
+level: DEBUG
+  celery:
+level: DEBUG
+  requests:
+level: INFO
+  revscoring.dependencies.dependent:
+level: INFO
+  stopit:
+level: ERROR
+
+handlers:
+  stdout:
+class: logging.StreamHandler
+formatter: basic_human
+stream: ext://sys.stdout
+  file:
+class: logging.FileHandler
+formatter: basic_human
+filename: /vagrant/logs/ores.log
+  syslog:
+class: logging.handlers.SysLogHandler
+formatter: basic_human
+address: /dev/log
+
+formatters:
+  basic_human:
+format: "%(asctime)s %(levelname)s %(name)s: %(message)s"
+
+  # Extra verbosity for untangling threads:
+  threaded:
+format: "s%(asctime)s %(levelname)s %(name)s [P%(process)d T%(thread)d]: 
%(message)s"
diff --git a/puppet/modules/ores/templates/systemd/ores-celery.erb 
b/puppet/modules/ores/templates/systemd/ores-celery.erb
index 7c51e06..21493e8 100644
--- a/puppet/modules/ores/templates/systemd/ores-celery.erb
+++ b/puppet/modules/ores/templates/systemd/ores-celery.erb
@@ -12,7 +12,7 @@
 Group=www-data
 SyslogIdentifier=ores-celery
 WorkingDirectory=<%= @ores_root %>
-ExecStart=<%= @venv_dir %>/bin/ores applications.celery
+ExecStart=<%= @venv_dir %>/bin/ores applications.celery --logging-config <%= 
@logging_config %>
 Restart=always
 
 [Install]
diff --git a/puppet/modules/ores/templates/systemd/ores-wsgi.erb 
b/puppet/modules/ores/templates/systemd/ores-wsgi.erb
index 4503085..afeb81b 100644
--- a/puppet/modules/ores/templates/systemd/ores-wsgi.erb
+++ b/puppet/modules/ores/templates/systemd/ores-wsgi.erb
@@ -12,7 +12,7 @@
 Group=www-data
 SyslogIdentifier=ores-wsgi
 WorkingDirectory=<%= @ores_root %>
-ExecStart=<%= @venv_dir %>/bin/ores applications.wsgi --port=<%= @port %> 
--processes=1
+ExecStart=<%= @venv_dir %>/bin/ores applications.wsgi --port=<%= @port %> 
--processes=1 --logging-config <%= @logging_config %>
 Restart=always
 
 [Install]

-- 
To view, visit https://gerrit.wikimedia.org/r/401781
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5042b25a17accbfb5ec430d4c3a7b92ea45336fb
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: stretch-migration
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[stretch-migration]: Remove CheckModelVersions.php from ores role

2018-01-03 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401782 )

Change subject: Remove CheckModelVersions.php from ores role
..

Remove CheckModelVersions.php from ores role

It's not needed anymore

Bug: T183468
Change-Id: I59d6c47e6126684bb78fae7a8ba559876007f4e9
---
M puppet/modules/role/manifests/ores.pp
1 file changed, 0 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/82/401782/1

diff --git a/puppet/modules/role/manifests/ores.pp 
b/puppet/modules/role/manifests/ores.pp
index ca7d059..73a4208 100644
--- a/puppet/modules/role/manifests/ores.pp
+++ b/puppet/modules/role/manifests/ores.pp
@@ -24,12 +24,6 @@
 },
 }
 
-mediawiki::maintenance { 'check ORES model versions':
-command => '/usr/local/bin/mwscript 
extensions/ORES/maintenance/CheckModelVersions.php --wiki=wiki',
-unless  => "/usr/bin/mysql -u root -p${::mysql::root_password} -e 
'select * from ores_model' wiki | /bin/grep -q 'damaging'",
-require => Mediawiki::Extension['ORES'],
-}
-
 # Ensure that the maintenance script does not run before the API is alive,
 # when ORES is installed locally via role::ores_service.
 # This is pretty horrible but seems to be the only way of avoiding cycles.

-- 
To view, visit https://gerrit.wikimedia.org/r/401782
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I59d6c47e6126684bb78fae7a8ba559876007f4e9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: stretch-migration
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[stretch-migration]: Make it possible to install existing python modules "editably"

2018-01-03 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401779 )

Change subject: Make it possible to install existing python modules "editably"
..

Make it possible to install existing python modules "editably"

This is a small change to make the pip rules refresh correctly when passing
an existing git repo to `pip install -e`.

Bug: T181850
Change-Id: I6f31ec47541370576f97f5512091189ff2cc7b48
---
M puppet/modules/virtualenv/manifests/package.pp
1 file changed, 9 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/79/401779/1

diff --git a/puppet/modules/virtualenv/manifests/package.pp 
b/puppet/modules/virtualenv/manifests/package.pp
index 539dfc2..581d3ea 100644
--- a/puppet/modules/virtualenv/manifests/package.pp
+++ b/puppet/modules/virtualenv/manifests/package.pp
@@ -9,7 +9,8 @@
 # == Parameters
 # [*path*]
 #   Path to already previously initialized virtualenv.
-#   The package will be available under $path/src/$python_module.
+#   For editable installs, the package will be available under
+#   $path/src/$python_module.
 #
 # [*package*]
 #   Pip package to install.  Default: $title
@@ -20,9 +21,11 @@
 #already installed in the virtualenv.  Default: $title
 #
 # [*editable*]
-#Intall package in editable mode (pip -e), ie. does a git clone into the
+#Install package in editable mode (pip -e), ie. does a git clone into the
 #main venv folder instead of installing a subset of the files only into
 #./lib. You probably need to use the 'package' parameter if you use this.
+#If the package parameter points to a local path, link to that source
+#rather than cloning the source into ${path}/src.
 #Default: false
 #
 define virtualenv::package (
@@ -42,10 +45,10 @@
 unless  => "${path}/bin/python -c 'import ${python_module}'",
 }
 exec { "pip_install_${python_module}_editable_in_${path}":
-command   => "${path}/bin/pip install -e ${package}",
-cwd   => $path,
-creates   => "${path}/src/${python_module}",
-subscribe => 
Exec["pip_install_${python_module}_dependencies_in_${path}"],
+command => "${path}/bin/pip install -e ${package}",
+cwd => $path,
+refreshonly => true,
+subscribe   => 
Exec["pip_install_${python_module}_dependencies_in_${path}"],
 }
 } else {
 exec { "pip_install_${python_module}_in_${path}":

-- 
To view, visit https://gerrit.wikimedia.org/r/401779
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6f31ec47541370576f97f5512091189ff2cc7b48
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: stretch-migration
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[stretch-migration]: Remove dangling reference to ORES checkModelVersions

2018-01-03 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401783 )

Change subject: Remove dangling reference to ORES checkModelVersions
..

Remove dangling reference to ORES checkModelVersions

Bug: T183468
Change-Id: I03a37164dcd38d74b6a91891d5c0c86689e139ec
---
M puppet/modules/role/manifests/ores.pp
1 file changed, 0 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/83/401783/1

diff --git a/puppet/modules/role/manifests/ores.pp 
b/puppet/modules/role/manifests/ores.pp
index 73a4208..c6ea753 100644
--- a/puppet/modules/role/manifests/ores.pp
+++ b/puppet/modules/role/manifests/ores.pp
@@ -23,10 +23,4 @@
 },
 },
 }
-
-# Ensure that the maintenance script does not run before the API is alive,
-# when ORES is installed locally via role::ores_service.
-# This is pretty horrible but seems to be the only way of avoiding cycles.
-Systemd::Service<| title == 'ores-wsgi' or title == 'ores-celery' |>
--> Mediawiki::Maintenance['check ORES model versions']
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/401783
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I03a37164dcd38d74b6a91891d5c0c86689e139ec
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: stretch-migration
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Remove dangling reference to ORES checkModelVersions

2018-01-03 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/401775 )

Change subject: Remove dangling reference to ORES checkModelVersions
..

Remove dangling reference to ORES checkModelVersions

Bug: T183468
Change-Id: I03a37164dcd38d74b6a91891d5c0c86689e139ec
---
M puppet/modules/role/manifests/ores.pp
1 file changed, 0 insertions(+), 7 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/75/401775/1

diff --git a/puppet/modules/role/manifests/ores.pp 
b/puppet/modules/role/manifests/ores.pp
index d912258..c6ea753 100644
--- a/puppet/modules/role/manifests/ores.pp
+++ b/puppet/modules/role/manifests/ores.pp
@@ -23,11 +23,4 @@
 },
 },
 }
-
-# Ensure that the maintenance script does not run before the API is alive,
-# when ORES is installed locally via role::ores_service.
-# This is pretty horrible but seems to be the only way of avoiding cycles.
-Systemd::Service<| title == 'ores-wsgi' or title == 'ores-celery' |>
-->
-Mediawiki::Maintenance['check ORES model versions']
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/401775
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I03a37164dcd38d74b6a91891d5c0c86689e139ec
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] translatewiki[master]: Point developers to the surprising group label.

2017-12-21 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/399676 )

Change subject: Point developers to the surprising group label.
..

Point developers to the surprising group label.

Change-Id: Id1cde734d3b2ce7899182f024f5ddf8ccf20d15c
---
M repoconfig.yaml
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/translatewiki 
refs/changes/76/399676/1

diff --git a/repoconfig.yaml b/repoconfig.yaml
index 6a196ca..aa88136 100644
--- a/repoconfig.yaml
+++ b/repoconfig.yaml
@@ -531,6 +531,8 @@
   url: https://github.com/lonvia/waymarked-trails-site.git
   url|export: g...@github.com:lonvia/waymarked-trails-site.git
 
+# Note that this group is called "Revision Scoring" in the English web
+# interface.
 wiki-ai:
   group: wiki-ai-*
   repos:

-- 
To view, visit https://gerrit.wikimedia.org/r/399676
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id1cde734d3b2ce7899182f024f5ddf8ccf20d15c
Gerrit-PatchSet: 1
Gerrit-Project: translatewiki
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump ores submodule

2017-12-21 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/399661 )

Change subject: Bump ores submodule
..


Bump ores submodule

commit 283f160ecdad61d5b91c206c20304dbc44d6bcf2
Merge: b8d5175 f3ed734
Author: Aaron Halfaker 
Date:   Thu Dec 21 10:00:20 2017 -0600

Merge pull request #243 from wiki-ai/console_logging

Always configure logging; log to console

commit f3ed734ba2c3cdbd4cfe0b8b5299ad5aff753850
Author: Adam Roses Wight 
Date:   Wed Dec 20 16:24:10 2017 -0500

Default to stderr for consistency with logging.basicConfig

commit 1eca3a723c83159cdf5266edd2e48ba6f09e2a10
Author: Adam Roses Wight 
Date:   Wed Dec 20 14:27:55 2017 -0500

Eat unexpected parameters with splat args

commit 4bfe7fc9e370075250575cde50537bdca22164be
Author: Adam Roses Wight 
Date:   Wed Dec 20 14:19:30 2017 -0500

Always configure logging; log to console

Push logging config down so it's executed from every entry point.

Detect whether we're running from the console, and mirror to console
if so.

If the --logging-config argument isn't given, check whether we can
find a distributed logging_config.yaml in the working directory.

Bug: https://phabricator.wikimedia.org/T182614

Change-Id: If34622d664ad7a68625ea049f39f3cf9b1caead7
---
M submodules/ores
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/submodules/ores b/submodules/ores
index b8d5175..283f160 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit b8d51750f3aeb7d99fc4ac375b8617b50199ddd4
+Subproject commit 283f160ecdad61d5b91c206c20304dbc44d6bcf2

-- 
To view, visit https://gerrit.wikimedia.org/r/399661
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: If34622d664ad7a68625ea049f39f3cf9b1caead7
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump ores submodule

2017-12-21 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/399661 )

Change subject: Bump ores submodule
..

Bump ores submodule

commit 283f160ecdad61d5b91c206c20304dbc44d6bcf2
Merge: b8d5175 f3ed734
Author: Aaron Halfaker 
Date:   Thu Dec 21 10:00:20 2017 -0600

Merge pull request #243 from wiki-ai/console_logging

Always configure logging; log to console

commit f3ed734ba2c3cdbd4cfe0b8b5299ad5aff753850
Author: Adam Roses Wight 
Date:   Wed Dec 20 16:24:10 2017 -0500

Default to stderr for consistency with logging.basicConfig

commit 1eca3a723c83159cdf5266edd2e48ba6f09e2a10
Author: Adam Roses Wight 
Date:   Wed Dec 20 14:27:55 2017 -0500

Eat unexpected parameters with splat args

commit 4bfe7fc9e370075250575cde50537bdca22164be
Author: Adam Roses Wight 
Date:   Wed Dec 20 14:19:30 2017 -0500

Always configure logging; log to console

Push logging config down so it's executed from every entry point.

Detect whether we're running from the console, and mirror to console
if so.

If the --logging-config argument isn't given, check whether we can
find a distributed logging_config.yaml in the working directory.

Bug: https://phabricator.wikimedia.org/T182614

Change-Id: If34622d664ad7a68625ea049f39f3cf9b1caead7
---
M submodules/ores
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/61/399661/1

diff --git a/submodules/ores b/submodules/ores
index b8d5175..283f160 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit b8d51750f3aeb7d99fc4ac375b8617b50199ddd4
+Subproject commit 283f160ecdad61d5b91c206c20304dbc44d6bcf2

-- 
To view, visit https://gerrit.wikimedia.org/r/399661
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If34622d664ad7a68625ea049f39f3cf9b1caead7
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Catch another double-quote, this may change the behavior of ...

2017-12-20 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/399465 )

Change subject: Catch another double-quote, this may change the behavior of the 
three changes feeds.
..

Catch another double-quote, this may change the behavior of the three changes 
feeds.

Change-Id: I9876698af12409ee3cbb8cd95876b9c3bb2342fd
---
M includes/Hooks.php
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/65/399465/1

diff --git a/includes/Hooks.php b/includes/Hooks.php
index 43c3ec5..cb6b549 100644
--- a/includes/Hooks.php
+++ b/includes/Hooks.php
@@ -384,7 +384,7 @@
return;
}
// FIXME: This is not a "filter" but an undocumented side 
effect of this function.
-   $fields['ores_damaging_threshold'] = $dbr->addQuotes( 
$threshold );
+   $fields['ores_damaging_threshold'] = $threshold;
 
if ( $hidenondamaging ) {
// Filter out non-damaging edits.

-- 
To view, visit https://gerrit.wikimedia.org/r/399465
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9876698af12409ee3cbb8cd95876b9c3bb2342fd
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Don't double-quote model version

2017-12-20 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/399464 )

Change subject: Don't double-quote model version
..

Don't double-quote model version

We were inserting extra quotes around model versions, so instead of "0.4.0", we
would have "'0.4.0'".

Note that this change will affect API output, which will no longer have the
version double and single quotes.  Nothing else should be affected because we
refer to the model-version by its primary key internally.

Bug: T183266
Bug: T181848
Change-Id: Ia6b04482761559730db33afb35d6e5f0dc192f1a
---
M maintenance/CheckModelVersions.php
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/64/399464/1

diff --git a/maintenance/CheckModelVersions.php 
b/maintenance/CheckModelVersions.php
index c5f365d..38b60a5 100644
--- a/maintenance/CheckModelVersions.php
+++ b/maintenance/CheckModelVersions.php
@@ -39,13 +39,13 @@
$dbw->upsert( 'ores_model',
[
'oresm_name' => $name,
-   'oresm_version' => $dbw->addQuotes( 
$info['version'] ),
+   'oresm_version' => $info['version'],
'oresm_is_current' => 1,
],
[ 'oresm_name', 'oresm_version' ],
[
'oresm_name' => $name,
-   'oresm_version' => $dbw->addQuotes( 
$info['version'] ),
+   'oresm_version' => $info['version'],
'oresm_is_current' => 1,
],
__METHOD__

-- 
To view, visit https://gerrit.wikimedia.org/r/399464
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia6b04482761559730db33afb35d6e5f0dc192f1a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump ores submodule

2017-12-19 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/399229 )

Change subject: Bump ores submodule
..


Bump ores submodule

Change-Id: I36e569f50e1c59cd4cfc8b24d923f4effb3a3ac4
---
M submodules/ores
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Halfak: Verified; Looks good to me, approved



diff --git a/submodules/ores b/submodules/ores
index a3a50bb..b8d5175 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit a3a50bb0b5aea50178e428430438ae3323d7376f
+Subproject commit b8d51750f3aeb7d99fc4ac375b8617b50199ddd4

-- 
To view, visit https://gerrit.wikimedia.org/r/399229
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I36e569f50e1c59cd4cfc8b24d923f4effb3a3ac4
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Awight 
Gerrit-Reviewer: Halfak 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Clean up logging config.

2017-12-19 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/398942 )

Change subject: Clean up logging config.
..


Clean up logging config.

Add modules we're interested in, as knobs to be turned when desired.

Make the format string nicer, and provide a variant for debugging problems
where threading would make it hard to read.

Bug: T182614
Change-Id: Ie6f36290d2afef2a77e44bbd5d36f0994cc65a1a
---
M logging_config.yaml
1 file changed, 23 insertions(+), 10 deletions(-)

Approvals:
  Halfak: Verified; Looks good to me, approved



diff --git a/logging_config.yaml b/logging_config.yaml
index aff222d..68cfc82 100644
--- a/logging_config.yaml
+++ b/logging_config.yaml
@@ -1,21 +1,26 @@
 version: 1
 
+root:
+  handlers: [file]
+
 loggers:
+
   revscoring:
 level: ERROR
-handlers: [file]
   ores:
 level: ERROR
-handlers: [file]
+  ores.metrics_collectors.logger:
+level: ERROR
   celery:
 level: ERROR
-handlers: [file]
+  requests:
+level: ERROR
+  revscoring.dependencies.dependent:
+level: ERROR
+  stopit:
+level: ERROR
 
 handlers:
-  stdout:
-class: logging.StreamHandler
-formatter: basic_human
-stream: ext://sys.stdout
   file:
 class: logging.handlers.TimedRotatingFileHandler
 filename: /srv/log/ores/app.log
@@ -23,7 +28,15 @@
 when: midnight
 backupCount: 7
 
-formatters:
-  basic_human: 
-format: "%(asctime)s %(levelname)s:%(name)s -- %(message)s"
+  syslog:
+class: logging.handlers.SysLogHandler
+formatter: basic_human
+address: /dev/log
 
+formatters:
+  basic_human:
+format: "%(asctime)s %(levelname)s %(name)s: %(message)s"
+
+  # Extra verbosity for untangling threads:
+  threaded:
+format: "s%(asctime)s %(levelname)s %(name)s [P%(process)d T%(thread)d]: 
%(message)s"

-- 
To view, visit https://gerrit.wikimedia.org/r/398942
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Ie6f36290d2afef2a77e44bbd5d36f0994cc65a1a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Awight 
Gerrit-Reviewer: Halfak 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump ores submodule

2017-12-19 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/399229 )

Change subject: Bump ores submodule
..

Bump ores submodule

Change-Id: I36e569f50e1c59cd4cfc8b24d923f4effb3a3ac4
---
M submodules/ores
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/29/399229/1

diff --git a/submodules/ores b/submodules/ores
index a3a50bb..b8d5175 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit a3a50bb0b5aea50178e428430438ae3323d7376f
+Subproject commit b8d51750f3aeb7d99fc4ac375b8617b50199ddd4

-- 
To view, visit https://gerrit.wikimedia.org/r/399229
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I36e569f50e1c59cd4cfc8b24d923f4effb3a3ac4
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Disable the ORES UI on beta wikidatawiki

2017-12-19 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/399211 )

Change subject: Disable the ORES UI on beta wikidatawiki
..

Disable the ORES UI on beta wikidatawiki

Bug: T183266
Change-Id: I72e581ebc64a3b34077035fdb0c955889ed7b1c8
---
M wmf-config/InitialiseSettings-labs.php
1 file changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/11/399211/1

diff --git a/wmf-config/InitialiseSettings-labs.php 
b/wmf-config/InitialiseSettings-labs.php
index a94e15c..4e9a406 100644
--- a/wmf-config/InitialiseSettings-labs.php
+++ b/wmf-config/InitialiseSettings-labs.php
@@ -701,6 +701,10 @@
'wmgUseORES' => [
'simplewiki' => true, // T181848
],
+   'wgOresUiEnabled' => [
+   'default' => true,
+   'wikidatawiki' => false,
+   ],
'wgOresFiltersThresholds' => [
'simplewiki' => [ // Follows enwiki
'damaging' => [

-- 
To view, visit https://gerrit.wikimedia.org/r/399211
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I72e581ebc64a3b34077035fdb0c955889ed7b1c8
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Clean up logging config.

2017-12-18 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/398942 )

Change subject: Clean up logging config.
..

Clean up logging config.

Add modules we're interested in, as knobs to be turned when desired.

Make the format string nicer, and provide a variant for debugging problems
where threading would make it hard to read.

Bug: T182614
Change-Id: Ie6f36290d2afef2a77e44bbd5d36f0994cc65a1a
---
M logging_config.yaml
1 file changed, 23 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/42/398942/1

diff --git a/logging_config.yaml b/logging_config.yaml
index aff222d..68cfc82 100644
--- a/logging_config.yaml
+++ b/logging_config.yaml
@@ -1,21 +1,26 @@
 version: 1
 
+root:
+  handlers: [file]
+
 loggers:
+
   revscoring:
 level: ERROR
-handlers: [file]
   ores:
 level: ERROR
-handlers: [file]
+  ores.metrics_collectors.logger:
+level: ERROR
   celery:
 level: ERROR
-handlers: [file]
+  requests:
+level: ERROR
+  revscoring.dependencies.dependent:
+level: ERROR
+  stopit:
+level: ERROR
 
 handlers:
-  stdout:
-class: logging.StreamHandler
-formatter: basic_human
-stream: ext://sys.stdout
   file:
 class: logging.handlers.TimedRotatingFileHandler
 filename: /srv/log/ores/app.log
@@ -23,7 +28,15 @@
 when: midnight
 backupCount: 7
 
-formatters:
-  basic_human: 
-format: "%(asctime)s %(levelname)s:%(name)s -- %(message)s"
+  syslog:
+class: logging.handlers.SysLogHandler
+formatter: basic_human
+address: /dev/log
 
+formatters:
+  basic_human:
+format: "%(asctime)s %(levelname)s %(name)s: %(message)s"
+
+  # Extra verbosity for untangling threads:
+  threaded:
+format: "s%(asctime)s %(levelname)s %(name)s [P%(process)d T%(thread)d]: 
%(message)s"

-- 
To view, visit https://gerrit.wikimedia.org/r/398942
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie6f36290d2afef2a77e44bbd5d36f0994cc65a1a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Tune down ORES worker counts on the stress testing cluster

2017-12-14 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/398316 )

Change subject: Tune down ORES worker counts on the stress testing cluster
..

Tune down ORES worker counts on the stress testing cluster

We're pressed against the memory limit and possibly saw an OOM on
ores1001.  I've tuned the celery workers down by about 10%, and
tried to set the other knobs accordingly.

Bug: T182249
Change-Id: I5ee47571f56be0daedb416c7d40c2e97be72a523
---
M hieradata/role/common/ores/stresstest.yaml
1 file changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/16/398316/1

diff --git a/hieradata/role/common/ores/stresstest.yaml 
b/hieradata/role/common/ores/stresstest.yaml
index 3e5d743..1b45cb2 100644
--- a/hieradata/role/common/ores/stresstest.yaml
+++ b/hieradata/role/common/ores/stresstest.yaml
@@ -2,10 +2,10 @@
 admin::groups:
 - ores-admin
 standard::has_ganglia: false
-profile::ores::web::workers: 230
+profile::ores::web::workers: 195
 profile::ores::web::redis_host: "ores1001.eqiad.wmnet"
-profile::ores::celery::workers: 150
-profile::ores::celery::queue_maxsize: 600
+profile::ores::celery::workers: 135
+profile::ores::celery::queue_maxsize: 540
 # The password is in the private store, this is here for completeness
 # profile::ores::web::redis_password: nothing
 profile::base::notifications_enabled: '0'

-- 
To view, visit https://gerrit.wikimedia.org/r/398316
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5ee47571f56be0daedb416c7d40c2e97be72a523
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Add Icelandic dictionary for ORES on iswiki

2017-12-13 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/398078 )

Change subject: Add Icelandic dictionary for ORES on iswiki
..

Add Icelandic dictionary for ORES on iswiki

Bug: T181099
Change-Id: I8d530766382da3467db640b94f721b8ad44e1f34
---
M modules/ores/manifests/base.pp
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/78/398078/1

diff --git a/modules/ores/manifests/base.pp b/modules/ores/manifests/base.pp
index 2a8d811..8fb1d7a 100644
--- a/modules/ores/manifests/base.pp
+++ b/modules/ores/manifests/base.pp
@@ -32,6 +32,7 @@
 'myspell-fr',
 'myspell-he',
 'myspell-hu',
+'aspell-is',
 'myspell-it',
 'myspell-lv',
 'myspell-nb',

-- 
To view, visit https://gerrit.wikimedia.org/r/398078
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8d530766382da3467db640b94f721b8ad44e1f34
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Adds iswiki and eswikiquote reverted models.

2017-12-13 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/397962 )

Change subject: Adds iswiki and eswikiquote reverted models.
..


Adds iswiki and eswikiquote reverted models.

Bug: T182719
Change-Id: I7cc630e5e776ddfc7f3eb31ca7bdc7328310d3d1
---
M config/00-main.yaml
M submodules/ores
2 files changed, 25 insertions(+), 1 deletion(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/config/00-main.yaml b/config/00-main.yaml
index 2f23569..20ed2aa 100644
--- a/config/00-main.yaml
+++ b/config/00-main.yaml
@@ -46,6 +46,7 @@
   - elwiki
   - eswikibooks
   - eswiki
+  - eswikiquote
   - etwiki
   - fawiki
   - fiwiki
@@ -55,6 +56,7 @@
   - hrwiki
   - huwiki
   - idwiki
+  - iswiki
   - itwiki
   - kowiki
   - nlwiki
@@ -168,6 +170,13 @@
 "on": ["edit"]
   goodfaith:
 "on": ["edit"]
+  eswikiquote:
+extractor: eswikiquote_api
+scorer_models:
+  reverted: eswikiquote_revert
+precache:
+  reverted:
+"on": ["edit"]
   etwiki:
 extractor: etwiki_api
 scorer_models:
@@ -244,6 +253,13 @@
 extractor: idwiki_api
 scorer_models:
   reverted: idwiki_revert
+precache:
+  reverted:
+"on": ["edit"]
+  iswiki:
+extractor: iswiki_api
+scorer_models:
+  reverted: iswiki_revert
 precache:
   reverted:
 "on": ["edit"]
@@ -428,6 +444,8 @@
 host: https://es.wikipedia.org
   eswikibooks_api:
 host: https://es.wikibooks.org
+  eswikiquote_api:
+host: https://es.wikiquote.org
   etwiki_api:
 host: https://et.wikipedia.org
   fawiki_api:
@@ -446,6 +464,8 @@
 host: https://hu.wikipedia.org
   idwiki_api:
 host: https://id.wikipedia.org
+  iswiki_api:
+host: https://is.wikipedia.org
   itwiki_api:
 host: https://it.wikipedia.org
   kowiki_api:
@@ -515,6 +535,8 @@
 model_file: 
submodules/editquality/models/eswikibooks.damaging.gradient_boosting.model
   eswikibooks_goodfaith:
 model_file: 
submodules/editquality/models/eswikibooks.goodfaith.gradient_boosting.model
+  eswikiquote_revert:
+model_file: 
submodules/editquality/models/eswikiquote.reverted.gradient_boosting.model
   etwiki_damaging:
 model_file: 
submodules/editquality/models/etwiki.damaging.gradient_boosting.model
   etwiki_goodfaith:
@@ -545,6 +567,8 @@
 model_file: submodules/editquality/models/huwiki.reverted.rf.model
   idwiki_revert:
 model_file: 
submodules/editquality/models/idwiki.reverted.gradient_boosting.model
+  iswiki_revert:
+model_file: 
submodules/editquality/models/iswiki.reverted.gradient_boosting.model
   itwiki_revert:
 model_file: 
submodules/editquality/models/itwiki.reverted.gradient_boosting.model
   jawiki_revert:
diff --git a/submodules/ores b/submodules/ores
index b7413d6..a3a50bb 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit b7413d6abf80688d44b42da4a7d3a4d1a80ca150
+Subproject commit a3a50bb0b5aea50178e428430438ae3323d7376f

-- 
To view, visit https://gerrit.wikimedia.org/r/397962
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7cc630e5e776ddfc7f3eb31ca7bdc7328310d3d1
Gerrit-PatchSet: 2
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Halfak 
Gerrit-Reviewer: Awight 
Gerrit-Reviewer: Ladsgroup 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: scap: Remove git_rev setting

2017-12-13 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/398067 )

Change subject: scap: Remove git_rev setting
..


scap: Remove git_rev setting

It ended up causing a number of issues making it particularly difficult
to deploy varied versions of submodules and causing issues with scap

Bug: T181661
Change-Id: I3758a9991640abe859ae7f45f3bc6fe00e485959
---
M scap/scap.cfg
1 file changed, 0 insertions(+), 1 deletion(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/scap/scap.cfg b/scap/scap.cfg
index fbd8a4a..98227d6 100755
--- a/scap/scap.cfg
+++ b/scap/scap.cfg
@@ -1,7 +1,6 @@
 [global]
 git_repo: ores/deploy
 git_deploy_dir: /srv/deployment
-git_rev: origin/master
 ssh_user: deploy-service
 server_groups: canary, worker, cluster, default
 dsh_targets: ores

-- 
To view, visit https://gerrit.wikimedia.org/r/398067
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I3758a9991640abe859ae7f45f3bc6fe00e485959
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Alexandros Kosiaris 
Gerrit-Reviewer: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Check out both ores and revscoring as editable modules

2017-12-11 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/397618 )

Change subject: Check out both ores and revscoring as editable modules
..

Check out both ores and revscoring as editable modules

Bug: T181850
Change-Id: Ifcaa471006a90b8e76118fbb4e91523656ae740d
---
M puppet/modules/ores/manifests/init.pp
1 file changed, 44 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/18/397618/1

diff --git a/puppet/modules/ores/manifests/init.pp 
b/puppet/modules/ores/manifests/init.pp
index 0a01fb3..5ea7579 100644
--- a/puppet/modules/ores/manifests/init.pp
+++ b/puppet/modules/ores/manifests/init.pp
@@ -26,31 +26,67 @@
 # revscoring
 require_package('python3-dev', 'g++', 'gfortran', 'liblapack-dev', 
'libopenblas-dev', 'libenchant-dev')
 
+file { $deploy_dir:
+ensure => directory,
+}
+
 # ORES (in a venv as it needs Python 3)
-virtualenv::environment { $deploy_dir:
+$venv_dir = "${deploy_dir}/venv"
+virtualenv::environment { $venv_dir:
 ensure  => present,
 owner   => $::share_owner,
 group   => $::share_group,
 python  => 'python3',
-require => Package['python3-dev', 'g++', 'gfortran', 'liblapack-dev', 
'libopenblas-dev', 'libenchant-dev'],
+require => [
+Package['python3-dev', 'g++', 'gfortran', 'liblapack-dev', 
'libopenblas-dev', 'libenchant-dev'],
+File[$deploy_dir],
+],
+}
+
+$src_root = "${deploy_dir}/src"
+file { $src_root:
+ensure => directory,
+}
+git::clone { 'revscoring':
+directory => "${src_root}/revscoring",
+remote=> "https://github.com/wiki-ai/revscoring;,
+require   => File[$src_root],
+}
+git::clone { 'ores':
+directory => "${src_root}/ores",
+remote=> "https://github.com/wiki-ai/ores;,
+require   => File[$src_root],
+}
+
+virtualenv::package { 'revscoring':
+package  => "${src_root}/revscoring",
+path => $venv_dir,
+editable => true,
+require  => [
+Virtualenv::Environment[$venv_dir],
+Git::Clone['revscoring'],
+],
 }
 virtualenv::package { 'ores[redis]':
-package  => 'git+https://github.com/wiki-ai/ores.git#egg=ores',
-path => $deploy_dir,
+package  => "${src_root}/ores",
+path => $venv_dir,
 editable => true,
-require  => Virtualenv::Environment[$deploy_dir],
+require  => [
+Virtualenv::Environment[$venv_dir],
+Virtualenv::Package['revscoring'],
+Git::Clone['ores'],
+],
 }
-$repo_dir = "${deploy_dir}/src/ores"
 
 apache::reverse_proxy { 'ores':
 port => $port,
 }
 
-$cfg_file = "${repo_dir}/config/999-vagrant.yaml"
+$cfg_file = "${src_root}/ores/config/999-vagrant.yaml"
 file { $cfg_file:
 ensure  => present,
 content => template('ores/ores.yaml.erb'),
-require => Virtualenv::Environment[$deploy_dir],
+require => Virtualenv::Package['ores[redis]'],
 }
 
 systemd::service { 'ores-wsgi':

-- 
To view, visit https://gerrit.wikimedia.org/r/397618
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifcaa471006a90b8e76118fbb4e91523656ae740d
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Make it possible to install existing python modules "editably"

2017-12-11 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/397617 )

Change subject: Make it possible to install existing python modules "editably"
..

Make it possible to install existing python modules "editably"

This is a small change to make the pip rules refresh correctly when passing
an existing git repo to `pip install -e`.

Bug: T181850
Change-Id: I6f31ec47541370576f97f5512091189ff2cc7b48
---
M puppet/modules/virtualenv/manifests/package.pp
1 file changed, 9 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/17/397617/1

diff --git a/puppet/modules/virtualenv/manifests/package.pp 
b/puppet/modules/virtualenv/manifests/package.pp
index 539dfc2..581d3ea 100644
--- a/puppet/modules/virtualenv/manifests/package.pp
+++ b/puppet/modules/virtualenv/manifests/package.pp
@@ -9,7 +9,8 @@
 # == Parameters
 # [*path*]
 #   Path to already previously initialized virtualenv.
-#   The package will be available under $path/src/$python_module.
+#   For editable installs, the package will be available under
+#   $path/src/$python_module.
 #
 # [*package*]
 #   Pip package to install.  Default: $title
@@ -20,9 +21,11 @@
 #already installed in the virtualenv.  Default: $title
 #
 # [*editable*]
-#Intall package in editable mode (pip -e), ie. does a git clone into the
+#Install package in editable mode (pip -e), ie. does a git clone into the
 #main venv folder instead of installing a subset of the files only into
 #./lib. You probably need to use the 'package' parameter if you use this.
+#If the package parameter points to a local path, link to that source
+#rather than cloning the source into ${path}/src.
 #Default: false
 #
 define virtualenv::package (
@@ -42,10 +45,10 @@
 unless  => "${path}/bin/python -c 'import ${python_module}'",
 }
 exec { "pip_install_${python_module}_editable_in_${path}":
-command   => "${path}/bin/pip install -e ${package}",
-cwd   => $path,
-creates   => "${path}/src/${python_module}",
-subscribe => 
Exec["pip_install_${python_module}_dependencies_in_${path}"],
+command => "${path}/bin/pip install -e ${package}",
+cwd => $path,
+refreshonly => true,
+subscribe   => 
Exec["pip_install_${python_module}_dependencies_in_${path}"],
 }
 } else {
 exec { "pip_install_${python_module}_in_${path}":

-- 
To view, visit https://gerrit.wikimedia.org/r/397617
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6f31ec47541370576f97f5512091189ff2cc7b48
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[STABLE]: Less verbose Celery logging

2017-12-09 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/396587 )

Change subject: Less verbose Celery logging
..


Less verbose Celery logging

We actually want these logs, but are working around an unexplained log
message routing issue.  Instead of writing to app.log as expected, we've
been logging to daemon.log, and filling the root partition.

Unknown why this config would write to daemon.log.

Bug: T182497
Change-Id: I5b79d32b168d915026d37518d27a0e7654f43001
---
M logging_config.yaml
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/logging_config.yaml b/logging_config.yaml
index 6b5cb0e..aff222d 100644
--- a/logging_config.yaml
+++ b/logging_config.yaml
@@ -8,7 +8,7 @@
 level: ERROR
 handlers: [file]
   celery:
-level: INFO
+level: ERROR
 handlers: [file]
 
 handlers:

-- 
To view, visit https://gerrit.wikimedia.org/r/396587
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I5b79d32b168d915026d37518d27a0e7654f43001
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: STABLE
Gerrit-Owner: Awight 
Gerrit-Reviewer: ArielGlenn 
Gerrit-Reviewer: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[STABLE]: Less verbose Celery logging

2017-12-09 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/396587 )

Change subject: Less verbose Celery logging
..

Less verbose Celery logging

We actually want these logs, but are working around an unexplained log
message routing issue.  Instead of writing to app.log as expected, we've
been logging to daemon.log, and filling the root partition.

Unknown why this config would write to daemon.log.

Bug: T182497
Change-Id: I5b79d32b168d915026d37518d27a0e7654f43001
---
M logging_config.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/87/396587/1

diff --git a/logging_config.yaml b/logging_config.yaml
index 6b5cb0e..aff222d 100644
--- a/logging_config.yaml
+++ b/logging_config.yaml
@@ -8,7 +8,7 @@
 level: ERROR
 handlers: [file]
   celery:
-level: INFO
+level: ERROR
 handlers: [file]
 
 handlers:

-- 
To view, visit https://gerrit.wikimedia.org/r/396587
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5b79d32b168d915026d37518d27a0e7654f43001
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: STABLE
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Less verbose Celery logging

2017-12-09 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/396586 )

Change subject: Less verbose Celery logging
..


Less verbose Celery logging

We actually want these logs, but are working around an unexplained log
message routing issue.  Instead of writing to app.log as expected, we've
been logging to daemon.log, and filling the root partition.

Unknown why this config would write to daemon.log.

Bug: T182497
Change-Id: I5b79d32b168d915026d37518d27a0e7654f43001
---
M logging_config.yaml
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/logging_config.yaml b/logging_config.yaml
index 6b5cb0e..aff222d 100644
--- a/logging_config.yaml
+++ b/logging_config.yaml
@@ -8,7 +8,7 @@
 level: ERROR
 handlers: [file]
   celery:
-level: INFO
+level: ERROR
 handlers: [file]
 
 handlers:

-- 
To view, visit https://gerrit.wikimedia.org/r/396586
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I5b79d32b168d915026d37518d27a0e7654f43001
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Less verbose Celery logging

2017-12-09 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/396586 )

Change subject: Less verbose Celery logging
..

Less verbose Celery logging

We actually want these logs, but are working around an unexplained log
message routing issue.  Instead of writing to app.log as expected, we've
been logging to daemon.log, and filling the root partition.

Unknown why this config would write to daemon.log.

Bug: T182497
Change-Id: I5b79d32b168d915026d37518d27a0e7654f43001
---
M logging_config.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/86/396586/1

diff --git a/logging_config.yaml b/logging_config.yaml
index 6b5cb0e..aff222d 100644
--- a/logging_config.yaml
+++ b/logging_config.yaml
@@ -8,7 +8,7 @@
 level: ERROR
 handlers: [file]
   celery:
-level: INFO
+level: ERROR
 handlers: [file]
 
 handlers:

-- 
To view, visit https://gerrit.wikimedia.org/r/396586
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5b79d32b168d915026d37518d27a0e7654f43001
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Refactor ORES uWSGI workers to use an absolute count

2017-12-07 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/396055 )

Change subject: Refactor ORES uWSGI workers to use an absolute count
..

Refactor ORES uWSGI workers to use an absolute count

This decouples us from the hardware.

Bug: T182249
Bug: T182332
Change-Id: Ib4445ad05c01f2b1161fc457b8db9feac2f5b973
---
M modules/ores/manifests/web.pp
1 file changed, 2 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/55/396055/1

diff --git a/modules/ores/manifests/web.pp b/modules/ores/manifests/web.pp
index 5378bf7..ff1a110 100644
--- a/modules/ores/manifests/web.pp
+++ b/modules/ores/manifests/web.pp
@@ -1,7 +1,7 @@
 # = Class: ores::web
 # Sets up a uwsgi based web server for ORES running python3
 class ores::web(
-$workers_per_core = 2,
+$uwsgi_workers = 45,
 $redis_host = '127.0.0.1',
 $redis_password = undef,
 $port = 8081,
@@ -23,7 +23,6 @@
 'ALL=(root) NOPASSWD: /usr/sbin/service celery-ores-worker *',
 ]
 
-$processes = $::processorcount * $workers_per_core
 service::uwsgi { 'ores':
 port=> $port,
 sudo_rules  => $sudo_rules,
@@ -35,7 +34,7 @@
 need-plugins  => 'python3,stats_pusher_statsd',
 venv  => $ores::base::venv_path,
 logformat => '[pid: %(pid)] %(addr) (%(user)) {%(vars) vars in 
%(pktsize) bytes} [%(ctime)] %(method) %(uri) => generated %(rsize) bytes in 
%(msecs) msecs (%(proto) %(status)) %(headers) headers in %(hsize) bytes 
(%(switches) switches on core %(core)) user agent "%(uagent)"',
-processes => $processes,
+processes => $uwsgi_workers,
 add-header=> 'Access-Control-Allow-Origin: *',
 max-requests  => 200,
 stats-push=> 
"statsd:${graphite_server}:8125,ores.${::hostname}.uwsgi",

-- 
To view, visit https://gerrit.wikimedia.org/r/396055
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib4445ad05c01f2b1161fc457b8db9feac2f5b973
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump editquality and ores submodules

2017-12-06 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/395855 )

Change subject: Bump editquality and ores submodules
..


Bump editquality and ores submodules

Change-Id: I81ffbf89bd3d6090e642d7481e9427501bd0c804
---
M submodules/editquality
M submodules/ores
2 files changed, 2 insertions(+), 2 deletions(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/submodules/editquality b/submodules/editquality
index ac3042e..15d5283 16
--- a/submodules/editquality
+++ b/submodules/editquality
@@ -1 +1 @@
-Subproject commit ac3042ee681242c64c078ca67f2dc4a4699b6bbd
+Subproject commit 15d5283b7422919d85203b5ba907027f9356e421
diff --git a/submodules/ores b/submodules/ores
index f3a4a63..b7413d6 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit f3a4a6394dbc231e8ddde80b23340f42fb4b7328
+Subproject commit b7413d6abf80688d44b42da4a7d3a4d1a80ca150

-- 
To view, visit https://gerrit.wikimedia.org/r/395855
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I81ffbf89bd3d6090e642d7481e9427501bd0c804
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump editquality and ores submodules

2017-12-06 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/395855 )

Change subject: Bump editquality and ores submodules
..

Bump editquality and ores submodules

Change-Id: I81ffbf89bd3d6090e642d7481e9427501bd0c804
---
M submodules/editquality
M submodules/ores
2 files changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/55/395855/1

diff --git a/submodules/editquality b/submodules/editquality
index ac3042e..15d5283 16
--- a/submodules/editquality
+++ b/submodules/editquality
@@ -1 +1 @@
-Subproject commit ac3042ee681242c64c078ca67f2dc4a4699b6bbd
+Subproject commit 15d5283b7422919d85203b5ba907027f9356e421
diff --git a/submodules/ores b/submodules/ores
index f3a4a63..b7413d6 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit f3a4a6394dbc231e8ddde80b23340f42fb4b7328
+Subproject commit b7413d6abf80688d44b42da4a7d3a4d1a80ca150

-- 
To view, visit https://gerrit.wikimedia.org/r/395855
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I81ffbf89bd3d6090e642d7481e9427501bd0c804
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Tune ORES Celery workers up a bit

2017-12-05 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/395608 )

Change subject: Tune ORES Celery workers up a bit
..

Tune ORES Celery workers up a bit

Looks like we can support more workers without running out of memory.

Bug: T169246
Change-Id: Ic5cd8786f40e308efe1b3ead6ea358cbafd9472c
---
M hieradata/role/common/ores/stresstest.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/08/395608/1

diff --git a/hieradata/role/common/ores/stresstest.yaml 
b/hieradata/role/common/ores/stresstest.yaml
index 56485f9..a64d5ad 100644
--- a/hieradata/role/common/ores/stresstest.yaml
+++ b/hieradata/role/common/ores/stresstest.yaml
@@ -2,7 +2,7 @@
 admin::groups:
 - ores-admin
 profile::ores::web::redis_host: "ores1001.eqiad.wmnet"
-profile::ores::celery::workers: 100
+profile::ores::celery::workers: 150
 profile::ores::celery::queue_maxsize: 200
 # The password is in the private store, this is here for completeness
 # profile::ores::web::redis_password: nothing

-- 
To view, visit https://gerrit.wikimedia.org/r/395608
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic5cd8786f40e308efe1b3ead6ea358cbafd9472c
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: Reduce the number of Celery workers for ORES

2017-12-05 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/395579 )

Change subject: Reduce the number of Celery workers for ORES
..

Reduce the number of Celery workers for ORES

See the estimation in https://phabricator.wikimedia.org/T169246#3813675
We were hitting OOM as the workers warmed up.  Guessing the amount of
memory a warm worker requires is turning out to be difficult.

Bug: T169246
Change-Id: Id4543469d2f107e3b84fe34ae54a606f02f2f129
---
M hieradata/role/common/ores/stresstest.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/79/395579/1

diff --git a/hieradata/role/common/ores/stresstest.yaml 
b/hieradata/role/common/ores/stresstest.yaml
index c73b65f..56485f9 100644
--- a/hieradata/role/common/ores/stresstest.yaml
+++ b/hieradata/role/common/ores/stresstest.yaml
@@ -2,7 +2,7 @@
 admin::groups:
 - ores-admin
 profile::ores::web::redis_host: "ores1001.eqiad.wmnet"
-profile::ores::celery::workers: 240
+profile::ores::celery::workers: 100
 profile::ores::celery::queue_maxsize: 200
 # The password is in the private store, this is here for completeness
 # profile::ores::web::redis_password: nothing

-- 
To view, visit https://gerrit.wikimedia.org/r/395579
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id4543469d2f107e3b84fe34ae54a606f02f2f129
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Add ORES filter thresholds for simplewiki

2017-12-04 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/395066 )

Change subject: Add ORES filter thresholds for simplewiki
..

Add ORES filter thresholds for simplewiki

Bug: T181848
Change-Id: I4109751a976408bba4211803111268b05babcaaf
---
M wmf-config/InitialiseSettings-labs.php
1 file changed, 16 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/66/395066/1

diff --git a/wmf-config/InitialiseSettings-labs.php 
b/wmf-config/InitialiseSettings-labs.php
index 3045515..383c120 100644
--- a/wmf-config/InitialiseSettings-labs.php
+++ b/wmf-config/InitialiseSettings-labs.php
@@ -692,5 +692,21 @@
'wmgUseORES' => [
'simplewiki' => true, // T181848
],
+   'wgOresFiltersThresholds' => [
+   'simplewiki' => [ // Follows enwiki
+   'damaging' => [
+   'likelygood' => [ 'min' => 0, 'max' => 
'recall_at_precision(min_precision=0.99)' ],
+   'maybebad' => [ 'min' => 
'recall_at_precision(min_precision=0.15)', 'max' => 1 ],
+   'likelybad' => [ 'min' => 
'recall_at_precision(min_precision=0.45)', 'max' => 1 ],
+   // verylikelybad uses default
+   ],
+   'goodfaith' => [
+   'likelygood' => [ 'min' => 
'recall_at_precision(min_precision=0.99)', 'max' => 1 ],
+   'maybebad' => [ 'min' => 0, 'max' => 
'recall_at_precision(min_precision=0.15)' ],
+   // likelybad uses default
+   'verylikelybad' => [ 'min' => 0, 'max' 
=> 'recall_at_precision(min_precision=0.9)' ],
+   ],
+   ],
+   ],
];
 } # wmflLabsSettings()

-- 
To view, visit https://gerrit.wikimedia.org/r/395066
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4109751a976408bba4211803111268b05babcaaf
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Enable ORES on simplewiki

2017-12-04 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/395059 )

Change subject: Enable ORES on simplewiki
..

Enable ORES on simplewiki

Bug: T181848
Change-Id: If4eeaa3d7619ede8da583f32545573b04f1a65ae
---
M wmf-config/InitialiseSettings-labs.php
M wmf-config/InitialiseSettings.php
2 files changed, 1 insertion(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/59/395059/1

diff --git a/wmf-config/InitialiseSettings-labs.php 
b/wmf-config/InitialiseSettings-labs.php
index 3045515..00c2683 100644
--- a/wmf-config/InitialiseSettings-labs.php
+++ b/wmf-config/InitialiseSettings-labs.php
@@ -688,9 +688,5 @@
'wgCommentTableSchemaMigrationStage' => [
'default' => MIGRATION_WRITE_BOTH,
],
-
-   'wmgUseORES' => [
-   'simplewiki' => true, // T181848
-   ],
];
 } # wmflLabsSettings()
diff --git a/wmf-config/InitialiseSettings.php 
b/wmf-config/InitialiseSettings.php
index c3468c4..9b8a4da 100644
--- a/wmf-config/InitialiseSettings.php
+++ b/wmf-config/InitialiseSettings.php
@@ -18975,6 +18975,7 @@
'ptwiki' => true, // T139692
'rowiki' => true, // T170723
'ruwiki' => true,
+   'simplewiki' => true, // T181848
'sqwiki' => true, // T170723
'trwiki' => true, // T139992
 

-- 
To view, visit https://gerrit.wikimedia.org/r/395059
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If4eeaa3d7619ede8da583f32545573b04f1a65ae
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Try simplewiki ORES on beta.

2017-12-04 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/395052 )

Change subject: Try simplewiki ORES on beta.
..

Try simplewiki ORES on beta.

Bug: T181848
Change-Id: Id55afcbaebf8e4b3d2c58936f04052c16f5ebaa3
---
M wmf-config/InitialiseSettings-labs.php
1 file changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/52/395052/1

diff --git a/wmf-config/InitialiseSettings-labs.php 
b/wmf-config/InitialiseSettings-labs.php
index 00c2683..3deda7e 100644
--- a/wmf-config/InitialiseSettings-labs.php
+++ b/wmf-config/InitialiseSettings-labs.php
@@ -688,5 +688,9 @@
'wgCommentTableSchemaMigrationStage' => [
'default' => MIGRATION_WRITE_BOTH,
],
+
+   'wmgUseORES' => [
+   ›⠀⠀⠀'simplewiki' => true, // T181848
+   ],
];
 } # wmflLabsSettings()

-- 
To view, visit https://gerrit.wikimedia.org/r/395052
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id55afcbaebf8e4b3d2c58936f04052c16f5ebaa3
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Limit to no more than 3 cached revisions

2017-12-04 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/395048 )

Change subject: Limit to no more than 3 cached revisions
..

Limit to no more than 3 cached revisions

Bug: T182013
Change-Id: I7eea3e19539a90571c3df14a8f012b54b2ca9e3a
---
M scap/scap.cfg
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/48/395048/1

diff --git a/scap/scap.cfg b/scap/scap.cfg
index fbd8a4a..150c473 100755
--- a/scap/scap.cfg
+++ b/scap/scap.cfg
@@ -13,6 +13,7 @@
 batch_size: 80
 restart_service_batch_size: 1
 max_failures: 1
+cache_revs: 3
 # Will be deprecated in the next scap release.
 keyholder_key: deploy_service
 

-- 
To view, visit https://gerrit.wikimedia.org/r/395048
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7eea3e19539a90571c3df14a8f012b54b2ca9e3a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Use enwiki models on simplewiki.

2017-12-04 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/394759 )

Change subject: Use enwiki models on simplewiki.
..


Use enwiki models on simplewiki.

Bug: T181848
Change-Id: I4f0b63d99a2f9f48b2906faa0ee00963a92be69b
---
M config/00-main.yaml
1 file changed, 13 insertions(+), 0 deletions(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/config/00-main.yaml b/config/00-main.yaml
index b921968..2f23569 100644
--- a/config/00-main.yaml
+++ b/config/00-main.yaml
@@ -63,6 +63,7 @@
   - ptwiki
   - rowiki
   - ruwiki
+  - simplewiki
   - sqwiki
   - svwiki
   - testwiki
@@ -318,6 +319,18 @@
 "on": ["edit"]
   goodfaith:
 "on": ["edit"]
+  simplewiki:
+extractor: simplewiki_api
+scorer_models:
+  wp10: enwiki_wp10
+  draftquality: enwiki_draftquality
+  damaging: enwiki_damaging
+  goodfaith: enwiki_goodfaith
+precache:
+  damaging:
+"on": ["edit"]
+  goodfaith:
+"on": ["edit"]
   sqwiki:
 extractor: sqwiki_api
 scorer_models:

-- 
To view, visit https://gerrit.wikimedia.org/r/394759
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I4f0b63d99a2f9f48b2906faa0ee00963a92be69b
Gerrit-PatchSet: 2
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Halfak 
Gerrit-Reviewer: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Less snowflaky installation for pylru and redis packages

2017-12-01 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/394654 )

Change subject: Less snowflaky installation for pylru and redis packages
..

Less snowflaky installation for pylru and redis packages

Bug: T181850
Change-Id: I57a48e748cb8d37ea2cc92597e42d800484cbe59
---
M puppet/modules/ores/manifests/init.pp
1 file changed, 8 insertions(+), 14 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/54/394654/1

diff --git a/puppet/modules/ores/manifests/init.pp 
b/puppet/modules/ores/manifests/init.pp
index 089d08d..f9c8828 100644
--- a/puppet/modules/ores/manifests/init.pp
+++ b/puppet/modules/ores/manifests/init.pp
@@ -28,24 +28,20 @@
 
 # ORES (in a venv as it needs Python 3)
 virtualenv::environment { $deploy_dir:
-ensure  => present,
-owner   => $::share_owner,
-group   => $::share_group,
-python  => 'python3',
+ensure   => present,
+owner=> $::share_owner,
+group=> $::share_group,
+python   => 'python3',
+packages => [
+'pylru',
+'redis',
+],
 require => Package['python3-dev', 'g++', 'gfortran', 'liblapack-dev', 
'libopenblas-dev', 'libenchant-dev'],
 }
 virtualenv::package { 'ores':
 package  => 'git+https://github.com/wiki-ai/ores.git#egg=ores',
 path => $deploy_dir,
 editable => true,
-}
-#FIXME this should happen as part of normal dependency management but for 
some reason it doesn't
-# pylru probably needs to be fixed in the revscoring pakcage, redis in ores
-exec { 'pip_install_revscoring_dependencies_hack':
-command => "curl 
https://raw.githubusercontent.com/wiki-ai/revscoring/master/requirements.txt | 
${deploy_dir}/bin/pip install pylru redis -r /dev/stdin",
-cwd => $deploy_dir,
-subscribe   => Virtualenv::Package['ores'],
-refreshonly => true,
 }
 $repo_dir = "${deploy_dir}/src/ores"
 
@@ -66,7 +62,6 @@
 require   => [
 VirtualEnv::Package['ores'],
 Class['mediawiki::ready_service'],
-Exec['pip_install_revscoring_dependencies_hack'],
 Apache::Site['ores'],
 ],
 subscribe => [
@@ -80,7 +75,6 @@
 require   => [
 VirtualEnv::Package['ores'],
 Class['mediawiki::ready_service'],
-Exec['pip_install_revscoring_dependencies_hack'],
 Apache::Site['ores'],
 ],
 subscribe => [

-- 
To view, visit https://gerrit.wikimedia.org/r/394654
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I57a48e748cb8d37ea2cc92597e42d800484cbe59
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump ores submodule

2017-11-29 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/394122 )

Change subject: Bump ores submodule
..


Bump ores submodule

Gets us the RevIdScorer statistics

Change-Id: I55fd7d3978b4133beb0dcfd5d8698084db46a190
---
M submodules/ores
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/submodules/ores b/submodules/ores
index 2c54f42..f3a4a63 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit 2c54f421755349c10b83da4ce35e6ce92d2bfd92
+Subproject commit f3a4a6394dbc231e8ddde80b23340f42fb4b7328

-- 
To view, visit https://gerrit.wikimedia.org/r/394122
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I55fd7d3978b4133beb0dcfd5d8698084db46a190
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump ores submodule

2017-11-29 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/394122 )

Change subject: Bump ores submodule
..

Bump ores submodule

Gets us the RevIdScorer statistics

Change-Id: I55fd7d3978b4133beb0dcfd5d8698084db46a190
---
M submodules/ores
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/22/394122/1

diff --git a/submodules/ores b/submodules/ores
index 2c54f42..f3a4a63 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit 2c54f421755349c10b83da4ce35e6ce92d2bfd92
+Subproject commit f3a4a6394dbc231e8ddde80b23340f42fb4b7328

-- 
To view, visit https://gerrit.wikimedia.org/r/394122
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I55fd7d3978b4133beb0dcfd5d8698084db46a190
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Protect Special:Contributions code from missing threshold

2017-11-29 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/394109 )

Change subject: Protect Special:Contributions code from missing threshold
..

Protect Special:Contributions code from missing threshold

Don't try to filter by threshold if none is available.

Bug: T179830
Change-Id: Ib3db6a57a2032c9b4ca3c505a56eb63defa12888
---
M includes/Hooks/ContributionsHooksHandler.php
1 file changed, 4 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/09/394109/1

diff --git a/includes/Hooks/ContributionsHooksHandler.php 
b/includes/Hooks/ContributionsHooksHandler.php
index 45721f2..9a7cb64 100644
--- a/includes/Hooks/ContributionsHooksHandler.php
+++ b/includes/Hooks/ContributionsHooksHandler.php
@@ -70,7 +70,10 @@
}
 
// Doesn't have ores score, skipping.
-   if ( !isset( $row->ores_damaging_score ) ) {
+   if (
+   !isset( $row->ores_damaging_score ) ||
+   !isset( $row->ores_damaging_threshold )
+   ) {
return;
}
 

-- 
To view, visit https://gerrit.wikimedia.org/r/394109
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib3db6a57a2032c9b4ca3c505a56eb63defa12888
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Remove unprovisioned servers

2017-11-29 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/393822 )

Change subject: Remove unprovisioned servers
..


Remove unprovisioned servers

Change-Id: I1187cc80a39a3f36e25e07d91c9e51e6fc6969ec
---
M scap/ores-cluster
1 file changed, 0 insertions(+), 9 deletions(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/scap/ores-cluster b/scap/ores-cluster
index 9afcac3..e989b70 100644
--- a/scap/ores-cluster
+++ b/scap/ores-cluster
@@ -7,12 +7,3 @@
 ores1007.eqiad.wmnet
 ores1008.eqiad.wmnet
 ores1009.eqiad.wmnet
-ores2001.codfw.wmnet
-ores2002.codfw.wmnet
-ores2003.codfw.wmnet
-ores2004.codfw.wmnet
-ores2005.codfw.wmnet
-ores2006.codfw.wmnet
-ores2007.codfw.wmnet
-ores2008.codfw.wmnet
-ores2009.codfw.wmnet

-- 
To view, visit https://gerrit.wikimedia.org/r/393822
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I1187cc80a39a3f36e25e07d91c9e51e6fc6969ec
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Alexandros Kosiaris 
Gerrit-Reviewer: Awight 
Gerrit-Reviewer: Halfak 
Gerrit-Reviewer: Ladsgroup 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Increase celery verbosity; use message format including time...

2017-11-29 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/394060 )

Change subject: Increase celery verbosity; use message format including 
timestamp
..

Increase celery verbosity; use message format including timestamp

Bug: T181538
Change-Id: I8c1d5c776e1c3437d0fbdf34d2c2b6a30b7e8d2c
---
M logging_config.yaml
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/60/394060/1

diff --git a/logging_config.yaml b/logging_config.yaml
index 780c806..6b5cb0e 100644
--- a/logging_config.yaml
+++ b/logging_config.yaml
@@ -8,7 +8,7 @@
 level: ERROR
 handlers: [file]
   celery:
-level: ERROR
+level: INFO
 handlers: [file]
 
 handlers:
@@ -19,6 +19,7 @@
   file:
 class: logging.handlers.TimedRotatingFileHandler
 filename: /srv/log/ores/app.log
+formatter: basic_human
 when: midnight
 backupCount: 7
 

-- 
To view, visit https://gerrit.wikimedia.org/r/394060
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8c1d5c776e1c3437d0fbdf34d2c2b6a30b7e8d2c
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[wmf/1.31.0-wmf.10]: Rate limit thresholds failures to once per (minute x model x...

2017-11-28 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393966 )

Change subject: Rate limit thresholds failures to once per (minute x model x 
wiki)
..

Rate limit thresholds failures to once per (minute x model x wiki)

Bug: T181567
Change-Id: I6dc4e6929cb48f4d1f060bf4ab6614db3a62d01f
---
M includes/Stats.php
1 file changed, 17 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/66/393966/1

diff --git a/includes/Stats.php b/includes/Stats.php
index 6f87383..777fb2c 100644
--- a/includes/Stats.php
+++ b/includes/Stats.php
@@ -19,6 +19,7 @@
 use MediaWiki\Logger\LoggerFactory;
 use MediaWiki\MediaWikiServices;
 use Psr\Log\LoggerInterface;
+use WANObjectCache;
 
 class Stats {
 
@@ -42,7 +43,7 @@
 * @param \WANObjectCache $cache
 * @param LoggerInterface $logger
 */
-   public function __construct( Api $api, \WANObjectCache $cache, 
LoggerInterface $logger ) {
+   public function __construct( Api $api, WANObjectCache $cache, 
LoggerInterface $logger ) {
$this->api = $api;
$this->cache = $cache;
$this->logger = $logger;
@@ -52,7 +53,7 @@
$config = $this->getFiltersConfig( $model );
// Skip if the model is unconfigured or set to false.
if ( $config ) {
-   $stats = $this->tryFetchStats( $model, $fromCache );
+   $stats = $this->fetchStats( $model, $fromCache );
// Skip if stats are empty.
if ( $stats !== false ) {
return $this->parseThresholds( $stats, $model );
@@ -84,23 +85,14 @@
return $config;
}
 
-   private function tryFetchStats( $model, $fromCache ) {
-   try {
-   return $this->fetchStats( $model, $fromCache );
-   } catch ( \RuntimeException $exception ) {
-   $this->logger->error( 'Failed to fetch ORES stats: ' . 
$exception->getMessage() );
-   return false;
-   }
-   }
-
private function fetchStats( $model, $fromCache ) {
global $wgOresCacheVersion;
if ( $fromCache ) {
$key = $this->cache->makeKey( 'ORES', 
'threshold_statistics', $model, $wgOresCacheVersion );
$result = $this->cache->getWithSetCallback(
$key,
-   \WANObjectCache::TTL_DAY,
-   function () use ( $model ) {
+   WANObjectCache::TTL_DAY,
+   function ( $oldValue, &$ttl, &$setOpts, $opts ) 
use ( $model ) {
$statsdDataFactory = 
MediaWikiServices::getInstance()->getStatsdDataFactory();
// @deprecated Only catching exceptions 
to allow the
// failure to be cached, remove once 
transition is
@@ -108,10 +100,18 @@
try {
$result = 
$this->fetchStatsFromApi( $model );
$statsdDataFactory->increment( 
'ores.api.stats.ok' );
+
return $result;
} catch ( \RuntimeException $ex ) {
+   // TODO: We can also check the 
service *before* the
+   // cached value expires, and 
therefore reuse the old
+   // value until the service 
recovers in case of failure.
$statsdDataFactory->increment( 
'ores.api.stats.failed' );
-   throw $ex;
+   $this->logger->error( 'Failed 
to fetch ORES stats.' );
+
+   // Retry again soon.
+   $ttl = 
WANObjectCache::TTL_MINUTE;
+   return [];
}
}
);
@@ -276,7 +276,9 @@
}
 
$stat = $config;
-   if ( $bound === 'max' && $statsData['false'][$stat] === null ) {
+   if ( !isset( $statsData['false'] ) || !isset( 
$statsData['true'] ) ) {
+   return null;
+   } elseif ( $bound === 'max' && $statsData['false'][$stat] === 
null ) {
return null;
} elseif ( $bound === 'max' && isset( 
$statsData['false'][$stat]['threshold'] ) ) {

[MediaWiki-commits] [Gerrit] mediawiki...ORES[wmf/1.31.0-wmf.8]: Rate limit thresholds failures to once per (minute x model x...

2017-11-28 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393960 )

Change subject: Rate limit thresholds failures to once per (minute x model x 
wiki)
..

Rate limit thresholds failures to once per (minute x model x wiki)

Bug: T181567
Change-Id: I6dc4e6929cb48f4d1f060bf4ab6614db3a62d01f
---
M includes/Stats.php
1 file changed, 17 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/60/393960/1

diff --git a/includes/Stats.php b/includes/Stats.php
index bc0f04e..cf27ecf 100644
--- a/includes/Stats.php
+++ b/includes/Stats.php
@@ -5,6 +5,7 @@
 use MediaWiki\Logger\LoggerFactory;
 use MediaWiki\MediaWikiServices;
 use Psr\Log\LoggerInterface;
+use WANObjectCache;
 
 class Stats {
 
@@ -28,7 +29,7 @@
 * @param \WANObjectCache $cache
 * @param LoggerInterface $logger
 */
-   public function __construct( Api $api, \WANObjectCache $cache, 
LoggerInterface $logger ) {
+   public function __construct( Api $api, WANObjectCache $cache, 
LoggerInterface $logger ) {
$this->api = $api;
$this->cache = $cache;
$this->logger = $logger;
@@ -38,7 +39,7 @@
$config = $this->getFiltersConfig( $model );
// Skip if the model is unconfigured or set to false.
if ( $config ) {
-   $stats = $this->tryFetchStats( $model, $fromCache );
+   $stats = $this->fetchStats( $model, $fromCache );
// Skip if stats are empty.
if ( $stats !== false ) {
return $this->parseThresholds( $stats, $model );
@@ -70,23 +71,14 @@
return $config;
}
 
-   private function tryFetchStats( $model, $fromCache ) {
-   try {
-   return $this->fetchStats( $model, $fromCache );
-   } catch ( \RuntimeException $exception ) {
-   $this->logger->error( 'Failed to fetch ORES stats: ' . 
$exception->getMessage() );
-   return false;
-   }
-   }
-
private function fetchStats( $model, $fromCache ) {
global $wgOresCacheVersion;
if ( $fromCache ) {
$key = $this->cache->makeKey( 'ORES', 
'threshold_statistics', $model, $wgOresCacheVersion );
$result = $this->cache->getWithSetCallback(
$key,
-   \WANObjectCache::TTL_DAY,
-   function () use ( $model ) {
+   WANObjectCache::TTL_DAY,
+   function ( $oldValue, &$ttl, &$setOpts, $opts ) 
use ( $model ) {
$statsdDataFactory = 
MediaWikiServices::getInstance()->getStatsdDataFactory();
// @deprecated Only catching exceptions 
to allow the
// failure to be cached, remove once 
transition is
@@ -94,10 +86,18 @@
try {
$result = 
$this->fetchStatsFromApi( $model );
$statsdDataFactory->increment( 
'ores.api.stats.ok' );
+
return $result;
} catch ( \RuntimeException $ex ) {
+   // TODO: We can also check the 
service *before* the
+   // cached value expires, and 
therefore reuse the old
+   // value until the service 
recovers in case of failure.
$statsdDataFactory->increment( 
'ores.api.stats.failed' );
-   throw $ex;
+   $this->logger->error( 'Failed 
to fetch ORES stats.' );
+
+   // Retry again soon.
+   $ttl = 
WANObjectCache::TTL_MINUTE;
+   return [];
}
}
);
@@ -262,7 +262,9 @@
}
 
$stat = $config;
-   if ( $bound === 'max' && $statsData['false'][$stat] === null ) {
+   if ( !isset( $statsData['false'] ) || !isset( 
$statsData['true'] ) ) {
+   return null;
+   } elseif ( $bound === 'max' && $statsData['false'][$stat] === 
null ) {
return null;
} elseif ( $bound === 'max' && isset( 
$statsData['false'][$stat]['threshold'] ) ) {

[MediaWiki-commits] [Gerrit] mediawiki...ORES[wmf/1.31.0-wmf.8]: Revert "Fallback to old thresholds API as necessary (take 2)"

2017-11-28 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393959 )

Change subject: Revert "Fallback to old thresholds API as necessary (take 2)"
..

Revert "Fallback to old thresholds API as necessary (take 2)"

Do not deploy until all old thresholds servers are replaced.

Simplifies thresholds handling by dropping back-compatibility with revscoring 1.

This mutatedly reverts commit 451b61af4a5a2e00426d18820009466a1c033812.

Bug: T179602
Change-Id: I455859cf6877efef3da318bfcf0884ddf84964f1
---
M extension.json
D includes/ApiV1.php
M includes/Stats.php
D includes/StatsV1.php
D tests/phpunit/includes/ApiV1Test.php
M tests/phpunit/includes/StatsTest.php
D tests/phpunit/includes/StatsV1Test.php
7 files changed, 4 insertions(+), 498 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/59/393959/1

diff --git a/extension.json b/extension.json
index 00fe46a..c3ce04a 100644
--- a/extension.json
+++ b/extension.json
@@ -10,7 +10,6 @@
"url": "https://www.mediawiki.org/wiki/Extension:ORES;,
"AutoloadClasses": {
"ORES\\Api": "includes/Api.php",
-   "ORES\\ApiV1": "includes/ApiV1.php",
"ORES\\Cache": "includes/Cache.php",
"ORES\\Hooks": "includes/Hooks.php",
"ORES\\Hooks\\ApiHooksHandler": 
"includes/Hooks/ApiHooksHandler.php",
@@ -19,7 +18,6 @@
"ORES\\Range": "includes/Range.php",
"ORES\\Scoring": "includes/Scoring.php",
"ORES\\Stats": "includes/Stats.php",
-   "ORES\\StatsV1": "includes/StatsV1.php",
"ORES\\ApiQueryORES": "includes/ApiQueryORES.php",
"ORES\\WatchedItemQueryServiceExtension": 
"includes/WatchedItemQueryServiceExtension.php"
},
diff --git a/includes/ApiV1.php b/includes/ApiV1.php
deleted file mode 100644
index aa93e3e..000
--- a/includes/ApiV1.php
+++ /dev/null
@@ -1,90 +0,0 @@
-setOriginalRequest( 
RequestContext::getMain()->getRequest() );
-   }
-   return $self;
-   }
-
-   /**
-* @param WebRequest|string[] $originalRequest See 
MwHttpRequest::setOriginalRequest()
-*/
-   public function setOriginalRequest( $originalRequest ) {
-   $this->originalRequest = $originalRequest;
-   }
-
-   /**
-* @param string|null $model Name of the model to query
-* @return string Base URL plus your wiki's `scores` API path.
-*/
-   public function getUrl( $model = null ) {
-   global $wgOresBaseUrl, $wgOresWikiId;
-
-   if ( $wgOresWikiId ) {
-   $wikiId = $wgOresWikiId;
-   } else {
-   $wikiId = wfWikiID();
-   }
-   $url = "{$wgOresBaseUrl}scores/{$wikiId}/";
-   if ( $model ) {
-   $url .= "{$model}/";
-   }
-   return $url;
-   }
-
-   /**
-* Make an ORES API request and return the decoded result.
-*
-* @param array $params optional GET parameters
-* @param string|null $model Name of the model to query
-* @return array Decoded response
-*
-*/
-   public function request( $params = [], $model = null ) {
-   $logger = LoggerFactory::getInstance( 'ORES' );
-
-   $url = $this->getUrl( $model );
-   $params['format'] = 'json';
-   $url = wfAppendQuery( $url, $params );
-   $logger->debug( "Requesting: {$url}" );
-   $req = MWHttpRequest::factory( $url, 
$this->getMWHttpRequestOptions(), __METHOD__ );
-   $status = $req->execute();
-   if ( !$status->isOK() ) {
-   throw new RuntimeException( "Failed to make ORES 
request to [{$url}], "
-   . $status->getMessage()->text() );
-   }
-   $json = $req->getContent();
-   $logger->debug( "Raw response: {$json}" );
-   $data = FormatJson::decode( $json, true );
-   if ( !$data || !empty( $data['error'] ) ) {
-   throw new RuntimeException( "Bad response from ORES 
endpoint [{$url}]: {$json}" );
-   }
-   return $data;
-   }
-
-   protected function getMWHttpRequestOptions() {
-   return $this->originalRequest ? [ 'originalRequest' => 
$this->originalRequest ] : [];
-   }
-
-}
diff --git a/includes/Stats.php b/includes/Stats.php
index 934426e..bc0f04e 100644
--- a/includes/Stats.php
+++ b/includes/Stats.php
@@ -97,16 +97,10 @@
return $result;
} catch ( \RuntimeException $ex ) {
$statsdDataFactory->increment(

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Cache anti-stampede improvements

2017-11-28 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393945 )

Change subject: Cache anti-stampede improvements
..

Cache anti-stampede improvements

Should prevent cache update race conditions.

Bug: T181567
Change-Id: I7d472cc23a45fd0a1c134b369ccd671cc1d5acee
---
M includes/Stats.php
1 file changed, 7 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/45/393945/1

diff --git a/includes/Stats.php b/includes/Stats.php
index 777fb2c..53872fc 100644
--- a/includes/Stats.php
+++ b/includes/Stats.php
@@ -113,7 +113,13 @@
$ttl = 
WANObjectCache::TTL_MINUTE;
return [];
}
-   }
+   },
+   [
+   // Try to only let one datacenter 
thread manage cache updates at a time
+   'lockTSE' => 10,
+   // Avoid querying cache servers 
multiple times in a web request
+   'pcTTL' => $cache::TTL_PROC_LONG,
+   ]
);
return $result;
} else {

-- 
To view, visit https://gerrit.wikimedia.org/r/393945
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7d472cc23a45fd0a1c134b369ccd671cc1d5acee
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Rate limit thresholds failures to once per (minute x model x...

2017-11-28 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393922 )

Change subject: Rate limit thresholds failures to once per (minute x model x 
wiki)
..

Rate limit thresholds failures to once per (minute x model x wiki)

Bug: T181567
Change-Id: I6dc4e6929cb48f4d1f060bf4ab6614db3a62d01f
---
M includes/Stats.php
1 file changed, 10 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/22/393922/1

diff --git a/includes/Stats.php b/includes/Stats.php
index 6f87383..e8e7582 100644
--- a/includes/Stats.php
+++ b/includes/Stats.php
@@ -52,7 +52,7 @@
$config = $this->getFiltersConfig( $model );
// Skip if the model is unconfigured or set to false.
if ( $config ) {
-   $stats = $this->tryFetchStats( $model, $fromCache );
+   $stats = $this->fetchStats( $model, $fromCache );
// Skip if stats are empty.
if ( $stats !== false ) {
return $this->parseThresholds( $stats, $model );
@@ -84,15 +84,6 @@
return $config;
}
 
-   private function tryFetchStats( $model, $fromCache ) {
-   try {
-   return $this->fetchStats( $model, $fromCache );
-   } catch ( \RuntimeException $exception ) {
-   $this->logger->error( 'Failed to fetch ORES stats: ' . 
$exception->getMessage() );
-   return false;
-   }
-   }
-
private function fetchStats( $model, $fromCache ) {
global $wgOresCacheVersion;
if ( $fromCache ) {
@@ -100,7 +91,7 @@
$result = $this->cache->getWithSetCallback(
$key,
\WANObjectCache::TTL_DAY,
-   function () use ( $model ) {
+   function ( $oldValue, &$ttl ) use ( $model ) {
$statsdDataFactory = 
MediaWikiServices::getInstance()->getStatsdDataFactory();
// @deprecated Only catching exceptions 
to allow the
// failure to be cached, remove once 
transition is
@@ -110,8 +101,15 @@
$statsdDataFactory->increment( 
'ores.api.stats.ok' );
return $result;
} catch ( \RuntimeException $ex ) {
+   // TODO: We can also check the 
service *before* the
+   // cached value expires, and 
therefore reuse the old
+   // value until the service 
recovers in case of failure.
$statsdDataFactory->increment( 
'ores.api.stats.failed' );
-   throw $ex;
+   $this->logger->error( 'Failed 
to fetch ORES stats.' );
+
+   // Retry again soon.
+   $ttl = 
\WANObjectCache::TTL_MINUTE;
+   return false;
}
}
);

-- 
To view, visit https://gerrit.wikimedia.org/r/393922
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6dc4e6929cb48f4d1f060bf4ab6614db3a62d01f
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: [WIP] Configure ORES logging

2017-11-28 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393831 )

Change subject: [WIP] Configure ORES logging
..

[WIP] Configure ORES logging

Change-Id: I5042b25a17accbfb5ec430d4c3a7b92ea45336fb
---
M puppet/modules/ores/manifests/init.pp
1 file changed, 9 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/31/393831/1

diff --git a/puppet/modules/ores/manifests/init.pp 
b/puppet/modules/ores/manifests/init.pp
index 089d08d..461d9b5 100644
--- a/puppet/modules/ores/manifests/init.pp
+++ b/puppet/modules/ores/manifests/init.pp
@@ -60,6 +60,13 @@
 require => Virtualenv::Package['ores'],
 }
 
+$logging_cfg = "${repo_dir}/logging_config.yaml"
+file { $logging_cfg:
+ensure  => present,
+content => template('ores/logging.yaml.erb'),
+require => Virtualenv::Package['ores'],
+}
+
 systemd::service { 'ores-wsgi':
 ensure => present,
 service_params => {
@@ -71,6 +78,7 @@
 ],
 subscribe => [
 File[$cfg_file],
+File[$logging_cfg],
 ],
 },
 }
@@ -85,6 +93,7 @@
 ],
 subscribe => [
 File[$cfg_file],
+File[$logging_cfg],
 ],
 },
 }

-- 
To view, visit https://gerrit.wikimedia.org/r/393831
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5042b25a17accbfb5ec430d4c3a7b92ea45336fb
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Remove unprovisioned servers

2017-11-28 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393822 )

Change subject: Remove unprovisioned servers
..

Remove unprovisioned servers

Change-Id: I1187cc80a39a3f36e25e07d91c9e51e6fc6969ec
---
M scap/ores-cluster
1 file changed, 0 insertions(+), 9 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/22/393822/1

diff --git a/scap/ores-cluster b/scap/ores-cluster
index 9afcac3..e989b70 100644
--- a/scap/ores-cluster
+++ b/scap/ores-cluster
@@ -7,12 +7,3 @@
 ores1007.eqiad.wmnet
 ores1008.eqiad.wmnet
 ores1009.eqiad.wmnet
-ores2001.codfw.wmnet
-ores2002.codfw.wmnet
-ores2003.codfw.wmnet
-ores2004.codfw.wmnet
-ores2005.codfw.wmnet
-ores2006.codfw.wmnet
-ores2007.codfw.wmnet
-ores2008.codfw.wmnet
-ores2009.codfw.wmnet

-- 
To view, visit https://gerrit.wikimedia.org/r/393822
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1187cc80a39a3f36e25e07d91c9e51e6fc6969ec
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Remove beta cluster customizations for ORES

2017-11-28 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393801 )

Change subject: Remove beta cluster customizations for ORES
..

Remove beta cluster customizations for ORES

We want the config to follow production except when explicitly testing new
settings.

Bug: T181187
Change-Id: I0bd82497cf04b89e6ee1cfb1826d8fa6fb7a3ede
---
M wmf-config/InitialiseSettings-labs.php
1 file changed, 0 insertions(+), 29 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/01/393801/1

diff --git a/wmf-config/InitialiseSettings-labs.php 
b/wmf-config/InitialiseSettings-labs.php
index 91d823f..e22b7dd 100644
--- a/wmf-config/InitialiseSettings-labs.php
+++ b/wmf-config/InitialiseSettings-labs.php
@@ -682,35 +682,6 @@
'wmgUseReadingLists' => [
'default' => true,
],
-   'wmgUseORES' => [
-   'ruwiki' => true, // T181168
-   ],
-   'wgOresModels' => [
-   'default' => [
-   'damaging' => true,
-   'goodfaith' => false,
-   'draftquality' => false,
-   'reverted' => false,
-   'wp10' => false,
-   ],
-   'enwiki' => [
-   'damaging' => true,
-   'goodfaith' => true,
-   'draftquality' => true,
-   'reverted' => false,
-   'wp10' => false,
-   ],
-   'ruwiki' => [
-   'damaging' => true,
-   'goodfaith' => true,
-   'draftquality' => false,
-   'reverted' => false,
-   'wp10' => false,
-   ],
-   ],
-   'wgOresDraftQualityNS' => [
-   'enwiki' => [ 0 => true, 118 => true ],
-   ],
 
// TODO(T166759): Remove this setting once it is also
// the default for production.

-- 
To view, visit https://gerrit.wikimedia.org/r/393801
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0bd82497cf04b89e6ee1cfb1826d8fa6fb7a3ede
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Reenable ORES on frwiki, ruwiki, and wikidatawiki

2017-11-27 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393667 )

Change subject: Reenable ORES on frwiki, ruwiki, and wikidatawiki
..

Reenable ORES on frwiki, ruwiki, and wikidatawiki

Bug: T181006
Change-Id: I239252057e88eb65e8e5aa226dba13c0ebf87b35
---
M wmf-config/InitialiseSettings.php
1 file changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/67/393667/1

diff --git a/wmf-config/InitialiseSettings.php 
b/wmf-config/InitialiseSettings.php
index 800b75c..461eaec 100644
--- a/wmf-config/InitialiseSettings.php
+++ b/wmf-config/InitialiseSettings.php
@@ -18954,17 +18954,17 @@
'etwiki' => true, // T159609
'fawiki' => true, // T130211
'fiwiki' => true, // T163011
-   'frwiki' => false, // T181006
+   'frwiki' => true,
'hewiki' => true, // T161621
'nlwiki' => true, // T139432
'plwiki' => true, // T140005
'ptwiki' => true, // T139692
'rowiki' => true, // T170723
-   'ruwiki' => false, // T181006
+   'ruwiki' => true,
'sqwiki' => true, // T170723
'trwiki' => true, // T139992
 
-   'wikidatawiki' => false, // T130212, temporarily disabled
+   'wikidatawiki' => true, // T130212
 ],
 'wgOresModels' => [
'default' => [

-- 
To view, visit https://gerrit.wikimedia.org/r/393667
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I239252057e88eb65e8e5aa226dba13c0ebf87b35
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[wmf/1.31.0-wmf.8]: Warn instead of throwing error in case threshold is unparseable

2017-11-27 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393660 )

Change subject: Warn instead of throwing error in case threshold is unparseable
..

Warn instead of throwing error in case threshold is unparseable

Bug: T181191
Change-Id: Ia0fe176b922efe1df10a1c310f88ccec32353562
---
M includes/Stats.php
1 file changed, 10 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/60/393660/1

diff --git a/includes/Stats.php b/includes/Stats.php
index 5c8a4bf..934426e 100644
--- a/includes/Stats.php
+++ b/includes/Stats.php
@@ -240,7 +240,15 @@
$statsData
);
 
-   if ( $max === null ) {
+   if ( $max === null || $min === null ) {
+   $data = [
+   'levelName' => $levelName,
+   'levelConfig' => $config,
+   'max' => $max,
+   'min' => $min,
+   'statsData' => $statsData,
+   ];
+   $this->logger->error( 'Unable to parse 
threshold: ' . json_encode( $data ) );
continue;
}
 
@@ -272,15 +280,7 @@
return $threshold;
}
 
-   throw new \RuntimeException(
-   'Unable to parse threshold: ' . json_encode(
-   [
-   'levelName' => $levelName,
-   'levelConfig' => $config,
-   'bound' => $bound,
-   'statsData' => $statsData,
-   ] )
-   );
+   return null;
}
 
/**

-- 
To view, visit https://gerrit.wikimedia.org/r/393660
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia0fe176b922efe1df10a1c310f88ccec32353562
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: wmf/1.31.0-wmf.8
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[wmf/1.31.0-wmf.8]: Disable the filter if ORES says the threshold doesn't exist

2017-11-27 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393659 )

Change subject: Disable the filter if ORES says the threshold doesn't exist
..

Disable the filter if ORES says the threshold doesn't exist

Bug: T181006
Change-Id: I35b13e9546ffd19279288fd4e1594a381faa6c06
---
M includes/Stats.php
1 file changed, 7 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/59/393659/1

diff --git a/includes/Stats.php b/includes/Stats.php
index f53f968..5c8a4bf 100644
--- a/includes/Stats.php
+++ b/includes/Stats.php
@@ -240,6 +240,10 @@
$statsData
);
 
+   if ( $max === null ) {
+   continue;
+   }
+
if ( is_numeric( $min ) && is_numeric( $max ) ) {
$thresholds[$levelName] = [
'min' => $min,
@@ -256,7 +260,9 @@
}
 
$stat = $config;
-   if ( $bound === 'max' && isset( 
$statsData['false'][$stat]['threshold'] ) ) {
+   if ( $bound === 'max' && $statsData['false'][$stat] === null ) {
+   return null;
+   } elseif ( $bound === 'max' && isset( 
$statsData['false'][$stat]['threshold'] ) ) {
$threshold = $statsData['false'][$stat]['threshold'];
// Invert to turn a "false" threshold to "true".
$threshold = 1 - $threshold;

-- 
To view, visit https://gerrit.wikimedia.org/r/393659
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I35b13e9546ffd19279288fd4e1594a381faa6c06
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: wmf/1.31.0-wmf.8
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/mediawiki-config[master]: Temporarily disable ORES on wikidata

2017-11-27 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393654 )

Change subject: Temporarily disable ORES on wikidata
..

Temporarily disable ORES on wikidata

This makes our deployment safer.

Change-Id: I0448e65c1371d3495702dc97954e9af7dc129bc5
---
M wmf-config/InitialiseSettings.php
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/mediawiki-config 
refs/changes/54/393654/1

diff --git a/wmf-config/InitialiseSettings.php 
b/wmf-config/InitialiseSettings.php
index 1903807..800b75c 100644
--- a/wmf-config/InitialiseSettings.php
+++ b/wmf-config/InitialiseSettings.php
@@ -18964,7 +18964,7 @@
'sqwiki' => true, // T170723
'trwiki' => true, // T139992
 
-   'wikidatawiki' => true, // T130212
+   'wikidatawiki' => false, // T130212, temporarily disabled
 ],
 'wgOresModels' => [
'default' => [

-- 
To view, visit https://gerrit.wikimedia.org/r/393654
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0448e65c1371d3495702dc97954e9af7dc129bc5
Gerrit-PatchSet: 1
Gerrit-Project: operations/mediawiki-config
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump editquality submodule

2017-11-27 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/393631 )

Change subject: Bump editquality submodule
..

Bump editquality submodule

Change-Id: I451f42a55242605af40e07b13e8fcf6053bacdf9
---
M submodules/editquality
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/31/393631/1

diff --git a/submodules/editquality b/submodules/editquality
index 789f51a..ac3042e 16
--- a/submodules/editquality
+++ b/submodules/editquality
@@ -1 +1 @@
-Subproject commit 789f51a06e0c06c5b0571c1deb71a06ecb7ba40b
+Subproject commit ac3042ee681242c64c078ca67f2dc4a4699b6bbd

-- 
To view, visit https://gerrit.wikimedia.org/r/393631
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I451f42a55242605af40e07b13e8fcf6053bacdf9
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] operations/puppet[production]: [DO NOT MERGE] Update ORES venv path to use versioned cache

2017-11-21 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/392683 )

Change subject: [DO NOT MERGE] Update ORES venv path to use versioned cache
..

[DO NOT MERGE] Update ORES venv path to use versioned cache

The virtualenv is now stored with

DNM because deployment is awkward.  We need to do something transitional,
perhaps as simple as creating a temporary symlink:
ln -sf deploy/venv /srv/deployment/ores/venv

Bug: T181071
Change-Id: I5b4df761047ef85b17e826b1ab8aecee6ef4eb26
---
M modules/ores/manifests/base.pp
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/operations/puppet 
refs/changes/83/392683/1

diff --git a/modules/ores/manifests/base.pp b/modules/ores/manifests/base.pp
index 2a8d811..d28c9e8 100644
--- a/modules/ores/manifests/base.pp
+++ b/modules/ores/manifests/base.pp
@@ -1,6 +1,6 @@
 class ores::base(
 $config_path = '/srv/deployment/ores/deploy',
-$venv_path = '/srv/deployment/ores/venv',
+$venv_path = '/srv/deployment/ores/deploy/venv',
 ) {
 # Let's use a virtualenv for maximum flexibility - we can convert
 # this to deb packages in the future if needed. We also install build tools

-- 
To view, visit https://gerrit.wikimedia.org/r/392683
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5b4df761047ef85b17e826b1ab8aecee6ef4eb26
Gerrit-PatchSet: 1
Gerrit-Project: operations/puppet
Gerrit-Branch: production
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: [WIP] Build venv into deployed source dir

2017-11-21 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/392682 )

Change subject: [WIP] Build venv into deployed source dir
..

[WIP] Build venv into deployed source dir

This moves the python library installation to the fetch stage, and stores
inside the source dir so we rollback faster.

TODO:
* Where do we specify the $venv_dir for services?  That needs to change.
* Why does the promote check only restart one service, and not ores-uwsgi?

Change-Id: Ibe44c2c3d7ec526a118a2b09a707616dd6cb3c05
---
M scap/checks.yaml
R scap/fetch_check.sh
A scap/promote_check.sh
3 files changed, 16 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/82/392682/1

diff --git a/scap/checks.yaml b/scap/checks.yaml
index d9eb373..b17dd02 100755
--- a/scap/checks.yaml
+++ b/scap/checks.yaml
@@ -1,25 +1,12 @@
 checks:
-  worker_checks:
+  fetch_checks:
+type: command
+stage: fetch
+timeout: 300
+command: bash /srv/deployment/ores/deploy/scap/fetch_check.sh
+
+  promote_checks:
 type: command
 stage: promote
 timeout: 300
-group: worker
-command: bash /srv/deployment/ores/deploy/scap/cmd_worker.sh
-  web_checks:
-type: command
-stage: promote
-group: web
-timeout: 300
-command: bash /srv/deployment/ores/deploy/scap/cmd_worker.sh
-  canary_checks:
-type: command
-stage: promote
-timeout: 300
-group: canary
-command: bash /srv/deployment/ores/deploy/scap/cmd_worker.sh
-  cluster_checks:
-type: command
-stage: promote
-timeout: 300
-group: cluster
-command: bash /srv/deployment/ores/deploy/scap/cmd_worker.sh
+command: bash /srv/deployment/ores/deploy/scap/promote_check.sh
diff --git a/scap/cmd_worker.sh b/scap/fetch_check.sh
similarity index 80%
rename from scap/cmd_worker.sh
rename to scap/fetch_check.sh
index 7ad3de5..47e5248 100644
--- a/scap/cmd_worker.sh
+++ b/scap/fetch_check.sh
@@ -1,11 +1,15 @@
-venv="/srv/deployment/ores/venv"
+# Point at deployed dirs.
 deploy_dir="/srv/deployment/ores/deploy"
+venv="${deploy_dir}/venv"
+
+# Pull submodules.
 cd $deploy_dir
 git submodule sync
 git submodule update --init
+
+# Install python libs.
 rm -rf $venv
 mkdir -p $venv
 virtualenv --python python3 --system-site-packages $venv
 [ -f $deploy_dir/submodules/wheels/pip-*.whl ] && $venv/bin/pip install 
--use-wheel --no-deps $deploy_dir/submodules/wheels/pip-*.whl
 $venv/bin/pip install --use-wheel --no-deps $deploy_dir/submodules/wheels/*.whl
-sudo service celery-ores-worker restart
diff --git a/scap/promote_check.sh b/scap/promote_check.sh
new file mode 100644
index 000..e96483e
--- /dev/null
+++ b/scap/promote_check.sh
@@ -0,0 +1,2 @@
+# Restart celery.
+sudo service celery-ores-worker restart

-- 
To view, visit https://gerrit.wikimedia.org/r/392682
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibe44c2c3d7ec526a118a2b09a707616dd6cb3c05
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Remove unused scripts

2017-11-21 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/392681 )

Change subject: Remove unused scripts
..

Remove unused scripts

I'm not 100% certain there isn't some magic that does actually call these.

Change-Id: Ie453700090379c3511393262cede4bbea234b3e0
---
D scap/cmd.sh
D scap/cmd_web.sh
2 files changed, 0 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/81/392681/1

diff --git a/scap/cmd.sh b/scap/cmd.sh
deleted file mode 100644
index 2f3a881..000
--- a/scap/cmd.sh
+++ /dev/null
@@ -1,6 +0,0 @@
-venv="/srv/deployment/ores/venv"
-deploy_dir="/srv/deployment/ores/deploy"
-mkdir -p $venv
-virtualenv --python python3 --system-site-packages $venv
-$venv/bin/pip freeze | xargs $venv/bin/pip uninstall -y
-$venv/bin/pip install --use-wheel --no-deps $deploy_dir/submodules/wheels/*.whl
diff --git a/scap/cmd_web.sh b/scap/cmd_web.sh
deleted file mode 100644
index 2f3a881..000
--- a/scap/cmd_web.sh
+++ /dev/null
@@ -1,6 +0,0 @@
-venv="/srv/deployment/ores/venv"
-deploy_dir="/srv/deployment/ores/deploy"
-mkdir -p $venv
-virtualenv --python python3 --system-site-packages $venv
-$venv/bin/pip freeze | xargs $venv/bin/pip uninstall -y
-$venv/bin/pip install --use-wheel --no-deps $deploy_dir/submodules/wheels/*.whl

-- 
To view, visit https://gerrit.wikimedia.org/r/392681
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie453700090379c3511393262cede4bbea234b3e0
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] research...wheels[master]: Remove old revscoring 2.0.9 wheel

2017-11-20 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/392459 )

Change subject: Remove old revscoring 2.0.9 wheel
..


Remove old revscoring 2.0.9 wheel

Change-Id: I4a82efb4ee94f500316fcc7924e9fe9d0bda2ce4
---
D revscoring-2.0.9-py2.py3-none-any.whl
1 file changed, 0 insertions(+), 0 deletions(-)

Approvals:
  jenkins-bot: Verified
  Awight: Looks good to me, approved



diff --git a/revscoring-2.0.9-py2.py3-none-any.whl 
b/revscoring-2.0.9-py2.py3-none-any.whl
deleted file mode 100644
index ea3716d..000
--- a/revscoring-2.0.9-py2.py3-none-any.whl
+++ /dev/null
Binary files differ

-- 
To view, visit https://gerrit.wikimedia.org/r/392459
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I4a82efb4ee94f500316fcc7924e9fe9d0bda2ce4
Gerrit-PatchSet: 1
Gerrit-Project: research/ores/wheels
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Awight 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Require revscoring 2.0.10

2017-11-20 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/392474 )

Change subject: Require revscoring 2.0.10
..

Require revscoring 2.0.10

Change-Id: Ic40117786bf109d1fe16d8ec5763841154923c6a
---
M requirements.txt
M submodules/wheels
2 files changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/74/392474/1

diff --git a/requirements.txt b/requirements.txt
index 6e5cc4a..5c4b279 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -3,6 +3,6 @@
 ores >= 1.0.0, < 1.0.999
 pip >= 9.0.1, < 9.0.999
 redis >= 2.10.5, < 2.10.999
-revscoring >= 2.0.8, < 2.0.999
+revscoring >= 2.0.10, < 2.0.999
 statsd >= 3.2.1, < 3.2.999
 wikiclass >= 0.3.2, < 0.3.999
diff --git a/submodules/wheels b/submodules/wheels
index 93e8846..d7fa640 16
--- a/submodules/wheels
+++ b/submodules/wheels
@@ -1 +1 @@
-Subproject commit 93e88468bf758c53dac5cade87262979fa9698f6
+Subproject commit d7fa640c59aebc3c080b43516c76235e1fca5cf9

-- 
To view, visit https://gerrit.wikimedia.org/r/392474
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic40117786bf109d1fe16d8ec5763841154923c6a
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] research...wheels[master]: Remove old revscoring 2.0.9 wheel

2017-11-20 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/392459 )

Change subject: Remove old revscoring 2.0.9 wheel
..

Remove old revscoring 2.0.9 wheel

Change-Id: I4a82efb4ee94f500316fcc7924e9fe9d0bda2ce4
---
D revscoring-2.0.9-py2.py3-none-any.whl
1 file changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/research/ores/wheels 
refs/changes/59/392459/1

diff --git a/revscoring-2.0.9-py2.py3-none-any.whl 
b/revscoring-2.0.9-py2.py3-none-any.whl
deleted file mode 100644
index ea3716d..000
--- a/revscoring-2.0.9-py2.py3-none-any.whl
+++ /dev/null
Binary files differ

-- 
To view, visit https://gerrit.wikimedia.org/r/392459
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4a82efb4ee94f500316fcc7924e9fe9d0bda2ce4
Gerrit-PatchSet: 1
Gerrit-Project: research/ores/wheels
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki/vagrant[master]: Use calculated thresholds

2017-11-15 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/391578 )

Change subject: Use calculated thresholds
..

Use calculated thresholds

Bug: T176914
Change-Id: I0bc4f2e57aa2b11226268b8883e05d3ffd8a3363
---
M puppet/modules/role/manifests/ores_service.pp
1 file changed, 5 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/vagrant 
refs/changes/78/391578/1

diff --git a/puppet/modules/role/manifests/ores_service.pp 
b/puppet/modules/role/manifests/ores_service.pp
index f5d646b..19296a0 100644
--- a/puppet/modules/role/manifests/ores_service.pp
+++ b/puppet/modules/role/manifests/ores_service.pp
@@ -17,28 +17,22 @@
 reverted  => false,
 wp10  => false,
 },
-# the mock scorer used by the local service cannot
-# provide recall stats so dynamic threshold configuration
-# would cause score fetches to silently fail. Use a
-# constant dummy configuration instead.
+# Use calculated thresholds.
 wgOresFiltersThresholds => {
 damaging => {
 likelygood=> {
 min => 0,
-max => 0.3,
+max => 'recall_at_precision(min_precision=0.99)',
 },
 maybebad  => {
-min => 0.2,
+min => 'recall_at_precision(min_precision=0.15)',
 max => 1,
 },
 likelybad => {
-min => 0.5,
+min => 'recall_at_precision(min_precision=0.45)',
 max => 1,
 },
-verylikelybad => {
-min => 0.8,
-max => 1,
-},
+# verylikelybad uses default
 },
 },
 },

-- 
To view, visit https://gerrit.wikimedia.org/r/391578
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0bc4f2e57aa2b11226268b8883e05d3ffd8a3363
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/vagrant
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Put all the wheel tools into the Makefile

2017-11-15 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/391562 )

Change subject: Put all the wheel tools into the Makefile
..

Put all the wheel tools into the Makefile

This lets us build wheels in prod and wmflabs as separate environments.

Corresponds to https://github.com/wiki-ai/ores-wmflabs-deploy/pull/91

Bug: T180496
Change-Id: I2a003a0209b3da37d8160520df33353588fae4dc
---
M Makefile
1 file changed, 26 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/62/391562/1

diff --git a/Makefile b/Makefile
index ab73dab..3144b0c 100644
--- a/Makefile
+++ b/Makefile
@@ -1,3 +1,28 @@
-deployment_wheels:
+REQUIREMENTS_FILES = \
+   submodules/ores/requirements.txt \
+   submodules/editquality/requirements.txt \
+   submodules/wikiclass/requirements.txt \
+   requirements.txt
+
+OMIT_WHEELS = \
+   ores \
+   editquality \
+   wikiclass \
+   setuptools \
+   pkg-resources
+
+pip_install:
+   pip install wheel
+   pip install --upgrade pip
+   for req_txt in $(REQUIREMENTS_FILES); do \
+ pip install -r $$req_txt; \
+   done
+
+frozen-requirements.txt: pip_install
+   pip freeze | \
+ grep -v $(addprefix -e , $(OMIT_WHEELS)) > \
+   frozen-requirements.txt
+
+deployment_wheels: frozen-requirements.txt
mkdir -p wheels && \
pip wheel -r frozen-requirements.txt -w wheels

-- 
To view, visit https://gerrit.wikimedia.org/r/391562
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2a003a0209b3da37d8160520df33353588fae4dc
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[CELERY_4]: Bump revscoring and ores

2017-11-14 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/391264 )

Change subject: Bump revscoring and ores
..

Bump revscoring and ores

FIXME: Still throws timeout errors 100% of the time.

Bug: T178441
Change-Id: I67e4d1a9db3aa5477440c6d161cf777ef1f8f484
---
M requirements.txt
M submodules/ores
M submodules/wheels
3 files changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/64/391264/1

diff --git a/requirements.txt b/requirements.txt
index eabf8b7..4f9808d 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -3,6 +3,6 @@
 ores >= 1.0.0, < 1.0.999
 pip >= 9.0.1, < 9.0.999
 redis >= 2.10.5, < 2.10.999
-revscoring >= 2.0.6, < 1.3.999
+revscoring >= 2.0.10, < 2.0.999
 statsd >= 3.2.1, < 3.2.999
 wikiclass >= 0.3.1, < 0.3.999
diff --git a/submodules/ores b/submodules/ores
index 1823595..a4a888e 16
--- a/submodules/ores
+++ b/submodules/ores
@@ -1 +1 @@
-Subproject commit 182359573bdeb359598ab33b3be7535ef0556d8d
+Subproject commit a4a888e235635c3682e4f18f1104ae5a91eeb683
diff --git a/submodules/wheels b/submodules/wheels
index 7343e35..7bf089d 16
--- a/submodules/wheels
+++ b/submodules/wheels
@@ -1 +1 @@
-Subproject commit 7343e350b61efadfbaf819b467ceb2a24a8b5307
+Subproject commit 7bf089d016d7c4b953226207d57fc3adfb2c1d67

-- 
To view, visit https://gerrit.wikimedia.org/r/391264
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I67e4d1a9db3aa5477440c6d161cf777ef1f8f484
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: CELERY_4
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] research...wheels[master]: Update revscoring wheel to 2.0.10

2017-11-14 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/391243 )

Change subject: Update revscoring wheel to 2.0.10
..

Update revscoring wheel to 2.0.10

Change-Id: If0f30359b2c4e7e94199571db4a321f093a74c60
---
A revscoring-2.0.10-py2.py3-none-any.whl
D revscoring-2.0.8-py2.py3-none-any.whl
2 files changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/research/ores/wheels 
refs/changes/43/391243/1

diff --git a/revscoring-2.0.10-py2.py3-none-any.whl 
b/revscoring-2.0.10-py2.py3-none-any.whl
new file mode 100644
index 000..5efe536
--- /dev/null
+++ b/revscoring-2.0.10-py2.py3-none-any.whl
Binary files differ
diff --git a/revscoring-2.0.8-py2.py3-none-any.whl 
b/revscoring-2.0.8-py2.py3-none-any.whl
deleted file mode 100644
index 951f35f..000
--- a/revscoring-2.0.8-py2.py3-none-any.whl
+++ /dev/null
Binary files differ

-- 
To view, visit https://gerrit.wikimedia.org/r/391243
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If0f30359b2c4e7e94199571db4a321f093a74c60
Gerrit-PatchSet: 1
Gerrit-Project: research/ores/wheels
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Merge master into wmf/1.31.0-wmf.7

2017-11-08 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/390069 )

Change subject: Merge master into wmf/1.31.0-wmf.7
..

Merge master into wmf/1.31.0-wmf.7

Change-Id: I0ddf1cdd3213e4282bcdb72467904b07489817bc
---
0 files changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/69/390069/1


-- 
To view, visit https://gerrit.wikimedia.org/r/390069
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0ddf1cdd3213e4282bcdb72467904b07489817bc
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...ORES[master]: Merge master into wmf/1.31.0-wmf.6

2017-11-08 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/390067 )

Change subject: Merge master into wmf/1.31.0-wmf.6
..

Merge master into wmf/1.31.0-wmf.6

Change-Id: I58cfbc967d0de985299d6d8e820ddf5ccfe974f8
---
0 files changed, 0 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/ORES 
refs/changes/67/390067/1


-- 
To view, visit https://gerrit.wikimedia.org/r/390067
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I58cfbc967d0de985299d6d8e820ddf5ccfe974f8
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/ORES
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump wikiclass

2017-11-07 Thread Awight (Code Review)

Awight has submitted this change and it was merged. ( 
https://gerrit.wikimedia.org/r/389756 )

Change subject: Bump wikiclass
..


Bump wikiclass

Bug: T179874
Change-Id: I364d1bd1a189087ebf59fe25c52d9ad00a0af6fd
---
M submodules/wikiclass
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  Awight: Verified; Looks good to me, approved



diff --git a/submodules/wikiclass b/submodules/wikiclass
index 5f4aa28..7810257 16
--- a/submodules/wikiclass
+++ b/submodules/wikiclass
@@ -1 +1 @@
-Subproject commit 5f4aa2840d6afcf247a9a3456663b87d392a85fd
+Subproject commit 78102570a2e1b68c98ab85932e67d34d3b51ddc3

-- 
To view, visit https://gerrit.wikimedia.org/r/389756
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I364d1bd1a189087ebf59fe25c52d9ad00a0af6fd
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 
Gerrit-Reviewer: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Bump wikiclass

2017-11-07 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/389756 )

Change subject: Bump wikiclass
..

Bump wikiclass

Bug: T179874
Change-Id: I364d1bd1a189087ebf59fe25c52d9ad00a0af6fd
---
M submodules/wikiclass
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/56/389756/1

diff --git a/submodules/wikiclass b/submodules/wikiclass
index 5f4aa28..7810257 16
--- a/submodules/wikiclass
+++ b/submodules/wikiclass
@@ -1 +1 @@
-Subproject commit 5f4aa2840d6afcf247a9a3456663b87d392a85fd
+Subproject commit 78102570a2e1b68c98ab85932e67d34d3b51ddc3

-- 
To view, visit https://gerrit.wikimedia.org/r/389756
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I364d1bd1a189087ebf59fe25c52d9ad00a0af6fd
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] mediawiki...deploy[master]: Correct default model class

2017-11-07 Thread Awight (Code Review)

Awight has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/389726 )

Change subject: Correct default model class
..

Correct default model class

Change-Id: I46b24a9899b2d4ae6020cdc307b5cb25c86729f4
---
M config/00-main.yaml
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/mediawiki/services/ores/deploy 
refs/changes/26/389726/1

diff --git a/config/00-main.yaml b/config/00-main.yaml
index d56a432..9b0bbae 100644
--- a/config/00-main.yaml
+++ b/config/00-main.yaml
@@ -471,7 +471,7 @@
 # Scorer models
 scorer_models:
   defaults:
-class: revscoring.scorer_models.MLScorerModel
+class: revscoring.scoring.Model
   arwiki_revert:
 model_file: 
submodules/editquality/models/arwiki.reverted.gradient_boosting.model
   bnwiki_revert:

-- 
To view, visit https://gerrit.wikimedia.org/r/389726
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I46b24a9899b2d4ae6020cdc307b5cb25c86729f4
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/services/ores/deploy
Gerrit-Branch: master
Gerrit-Owner: Awight 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

1 2 3 4 5 6 7 8 9 10 >

1 - 100 of 2839 matches

Mail list logo