from:"XZise \(Code Review\)"

[MediaWiki-commits] [Gerrit] integration/commit-message-validator[master]: Use one logic to catch all wrong spellings

2017-08-22 Thread XZise (Code Review)

XZise has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/373092 )

Change subject: Use one logic to catch all wrong spellings
..

Use one logic to catch all wrong spellings

Instead of having a custom logic for each wrong footer name section, it can
convert the normalized name into the correct spelling once and then check if
the given spelling matches that. Otherwise it shows a generic error.

Change-Id: I954b8808ca5f8a42e022a80817274275f104941b
---
M commit_message_validator/__init__.py
M commit_message_validator/tests/data/co-authored-by_ok.msg
2 files changed, 19 insertions(+), 23 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/integration/commit-message-validator 
refs/changes/92/373092/1

diff --git a/commit_message_validator/__init__.py 
b/commit_message_validator/__init__.py
index 7dfd41a..0b63eb8 100644
--- a/commit_message_validator/__init__.py
+++ b/commit_message_validator/__init__.py
@@ -39,20 +39,21 @@
 RE_CHERRYPICK = re.compile(r'^\(cherry picked from commit [0-9a-fA-F]{40}\)$')
 
 # Header-like lines that we are interested in validating
-FOOTERS = [
-'acked-by',
-'bug',
-'cc',
-'change-id',
-'co-authored-by',
-'depends-on',
-'reported-by',
-'reviewed-by',
-'signed-off-by',
-'suggested-by',
-'tested-by',
-'thanks',
+CORRECT_FOOTERS = [
+'Acked-By',
+'Bug',
+'Cc',
+'Change-Id',
+'Co-Authored-By',
+'Depends-On',
+'Reported-By',
+'Reviewed-by',
+'Signed-off-by',
+'Suggested-By',
+'Tested-by',
+'Thanks',
 ]
+FOOTERS = dict((footer.lower(), footer) for footer in CORRECT_FOOTERS)
 
 BEFORE_CHANGE_ID = [
 'bug',
@@ -149,21 +150,19 @@
 elif not self._in_footers:
 yield "Expected '{0}:' to be in footer".format(name)
 
+correct_name = FOOTERS[normalized_name]
+if correct_name != name:
+yield "Use '{0}:' not '{1}:'".format(correct_name, name)
+
 if normalized_name == 'bug':
-if name != 'Bug':
-yield "Use 'Bug:' not '{0}:'".format(name)
 if not is_valid_bug_id(value):
 yield "Bug: value must be a single phabricator task ID"
 
 elif normalized_name == 'depends-on':
-if name != 'Depends-On':
-yield "Use 'Depends-On:' not '%s:'" % name
 if not is_valid_change_id(value):
 yield "Depends-On: value must be a single Gerrit change id"
 
 elif normalized_name == 'change-id':
-if name != 'Change-Id':
-yield "Use 'Change-Id:' not '%s:'" % name
 if not is_valid_change_id(value):
 yield "Change-Id: value must be a single Gerrit change id"
 if self._first_changeid is not False:
@@ -171,9 +170,6 @@
"{0}".format(self._first_changeid))
 else:
 self._first_changeid = lineno + 1
-
-elif name[0].upper() != name[0]:
-yield "'%s:' must start with a capital letter" % name
 
 if (normalized_name in BEFORE_CHANGE_ID and
 self._first_changeid is not False):
diff --git a/commit_message_validator/tests/data/co-authored-by_ok.msg 
b/commit_message_validator/tests/data/co-authored-by_ok.msg
index 09f916f..fd170bc 100644
--- a/commit_message_validator/tests/data/co-authored-by_ok.msg
+++ b/commit_message_validator/tests/data/co-authored-by_ok.msg
@@ -6,5 +6,5 @@
 display with an 8 character tab leading the line) except in the case of a long
 URL that is embedded in the message.
 
-Co-authored-by: Someone awesome 
+Co-Authored-By: Someone awesome 
 Change-Id: Ifcd397165df1cbf9fa04f2044e1bb33ad7414d8d

-- 
To view, visit https://gerrit.wikimedia.org/r/373092
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I954b8808ca5f8a42e022a80817274275f104941b
Gerrit-PatchSet: 1
Gerrit-Project: integration/commit-message-validator
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] integration/commit-message-validator[master]: Fix actual tested minimum required number of lines

2017-08-01 Thread XZise (Code Review)

XZise has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/369499 )

Change subject: Fix actual tested minimum required number of lines
..

Fix actual tested minimum required number of lines

From the first implementation, it wouldn't fail when the commit message only
contained two lines, even if the requirement was that it has at least three
lines.

Change-Id: I8f8ce8acc540bff4a8e8759c57d53969cae91a6b
---
M commit_message_validator/__init__.py
A commit_message_validator/tests/data/short_one_line.msg
A commit_message_validator/tests/data/short_one_line.out
A commit_message_validator/tests/data/short_two_lines.msg
A commit_message_validator/tests/data/short_two_lines.out
5 files changed, 17 insertions(+), 1 deletion(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/integration/commit-message-validator 
refs/changes/99/369499/1

diff --git a/commit_message_validator/__init__.py 
b/commit_message_validator/__init__.py
index 848d61b..2e9ca8c 100644
--- a/commit_message_validator/__init__.py
+++ b/commit_message_validator/__init__.py
@@ -220,7 +220,7 @@
 last_lineno = rline
 last_line = line
 
-if last_lineno < 2:
+if last_lineno < 3:
 errors.append("Line %d: Expected at least 3 lines" % last_lineno)
 
 if changeid_line is False:
diff --git a/commit_message_validator/tests/data/short_one_line.msg 
b/commit_message_validator/tests/data/short_one_line.msg
new file mode 100644
index 000..9214fe6
--- /dev/null
+++ b/commit_message_validator/tests/data/short_one_line.msg
@@ -0,0 +1 @@
+Only one line
diff --git a/commit_message_validator/tests/data/short_one_line.out 
b/commit_message_validator/tests/data/short_one_line.out
new file mode 100644
index 000..9892d53
--- /dev/null
+++ b/commit_message_validator/tests/data/short_one_line.out
@@ -0,0 +1,6 @@
+commit-message-validator v%version%
+The following errors were found:
+Line 1: Expected at least 3 lines
+Line 1: Expected Change-Id
+Please review 
+and update your commit message accordingly
diff --git a/commit_message_validator/tests/data/short_two_lines.msg 
b/commit_message_validator/tests/data/short_two_lines.msg
new file mode 100644
index 000..de48893
--- /dev/null
+++ b/commit_message_validator/tests/data/short_two_lines.msg
@@ -0,0 +1,2 @@
+Only two lines
+Change-Id: Ifcd397165df1cbf9fa04f2044e1bb33ad7414d8d
diff --git a/commit_message_validator/tests/data/short_two_lines.out 
b/commit_message_validator/tests/data/short_two_lines.out
new file mode 100644
index 000..87935fd
--- /dev/null
+++ b/commit_message_validator/tests/data/short_two_lines.out
@@ -0,0 +1,7 @@
+commit-message-validator v%version%
+The following errors were found:
+Line 2: Second line should be empty
+Line 2: Expected 'Change-Id:' to be in footer
+Line 2: Expected at least 3 lines
+Please review 
+and update your commit message accordingly

-- 
To view, visit https://gerrit.wikimedia.org/r/369499
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8f8ce8acc540bff4a8e8759c57d53969cae91a6b
Gerrit-PatchSet: 1
Gerrit-Project: integration/commit-message-validator
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] integration/commit-message-validator[master]: Not always is the last line empty

2017-08-01 Thread XZise (Code Review)

XZise has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/369496 )

Change subject: Not always is the last line empty
..

Not always is the last line empty

In some instances the last line in a commit message is not empty, but it would
always discard the last line which could cause actually invalid failures.

Change-Id: I9a775d491ba7409d3154c821973f7652fc6993ff
---
M commit_message_validator/__init__.py
1 file changed, 4 insertions(+), 2 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/integration/commit-message-validator 
refs/changes/96/369496/1

diff --git a/commit_message_validator/__init__.py 
b/commit_message_validator/__init__.py
index 848d61b..8848e92 100644
--- a/commit_message_validator/__init__.py
+++ b/commit_message_validator/__init__.py
@@ -266,8 +266,10 @@
 
 commit = check_output(
 ['git', 'log', '--format=%B', '--no-color', commit_id, '-n1'])
-# last line is always an empty line
-lines = commit.splitlines()[:-1]
+lines = commit.splitlines()
+# last line is sometimes an empty line
+if len(lines) > 0 and not lines[-1]:
+lines = lines[:-1]
 
 return check_message(lines)
 

-- 
To view, visit https://gerrit.wikimedia.org/r/369496
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9a775d491ba7409d3154c821973f7652fc6993ff
Gerrit-PatchSet: 1
Gerrit-Project: integration/commit-message-validator
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] integration/commit-message-validator[master]: Decode text as UTF8

2017-08-01 Thread XZise (Code Review)

XZise has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/369472 )

Change subject: Decode text as UTF8
..

Decode text as UTF8

In Python 2 it by default uses ASCII to decode the command output, but that
will fail if the commit message contains non-ASCII characters. The next best
choice is to assume UTF8 will work.

Change-Id: Ib81b57212147a51b482703798cfe355946b915a6
---
M commit_message_validator/__init__.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/integration/commit-message-validator 
refs/changes/72/369472/1

diff --git a/commit_message_validator/__init__.py 
b/commit_message_validator/__init__.py
index 848d61b..7b05a40 100644
--- a/commit_message_validator/__init__.py
+++ b/commit_message_validator/__init__.py
@@ -248,7 +248,7 @@
 
 def check_output(args):
 """Wrapper around subprocess to handle Python 3"""
-return subprocess.check_output(args).decode()
+return subprocess.check_output(args).decode("utf8")
 
 
 def main(commit_id='HEAD'):

-- 
To view, visit https://gerrit.wikimedia.org/r/369472
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib81b57212147a51b482703798cfe355946b915a6
Gerrit-PatchSet: 1
Gerrit-Project: integration/commit-message-validator
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] integration/commit-message-validator[master]: Use float division for the percentage

2017-07-29 Thread XZise (Code Review)

XZise has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/368546 )

Change subject: Use float division for the percentage
..

Use float division for the percentage

The percentage of bad commits was using the integer division by default in
Python 2. With Python 3 the single slash will always use a float division and
importing division from __future__ backports that to Python 2. This is fixing
the percentage always showing 0 %.

Change-Id: I2ef4a0c0dc8295a8e5633cd4731120b09d02221e
---
M commit_message_validator/tests/sample_repository.py
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/integration/commit-message-validator 
refs/changes/46/368546/1

diff --git a/commit_message_validator/tests/sample_repository.py 
b/commit_message_validator/tests/sample_repository.py
index c786265..c3445a9 100644
--- a/commit_message_validator/tests/sample_repository.py
+++ b/commit_message_validator/tests/sample_repository.py
@@ -5,6 +5,7 @@
 messages to see whether they would pass the commit message
 validator.
 """
+from __future__ import division
 
 import os
 import sys

-- 
To view, visit https://gerrit.wikimedia.org/r/368546
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2ef4a0c0dc8295a8e5633cd4731120b09d02221e
Gerrit-PatchSet: 1
Gerrit-Project: integration/commit-message-validator
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] integration/commit-message-validator[master]: [IMPROV] Don't include BAD_FOOTERS in FOOTERS

2017-07-28 Thread XZise (Code Review)

XZise has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/368515 )

Change subject: [IMPROV] Don't include BAD_FOOTERS in FOOTERS
..

[IMPROV] Don't include BAD_FOOTERS in FOOTERS

If a key in BAD_FOOTERS is not included in FOOTERS it won't be tested, so it's
better to first normalize the footer further by using BAD_FOOTERS and after
that use FOOTERS.

Change-Id: I9353d07d41a2d4428189e0cc6b0b06a324c817ce
---
M commit_message_validator/__init__.py
1 file changed, 4 insertions(+), 7 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/integration/commit-message-validator 
refs/changes/15/368515/1

diff --git a/commit_message_validator/__init__.py 
b/commit_message_validator/__init__.py
index 4aaf665..48c9f5e 100644
--- a/commit_message_validator/__init__.py
+++ b/commit_message_validator/__init__.py
@@ -43,15 +43,12 @@
 'bug',
 'cc',
 'change-id',
-'closes',
 'co-authored-by',
 'depends-on',
-'fixes',
 'reported-by',
 'reviewed-by',
 'signed-off-by',
 'suggested-by',
-'task',
 'tested-by',
 'thanks',
 ]
@@ -135,6 +132,10 @@
 ws = m.group('ws')
 value = m.group('value')
 
+if normalized_name in BAD_FOOTERS:
+# Treat as the correct name for the rest of the rules
+normalized_name = BAD_FOOTERS[normalized_name]
+
 if normalized_name not in FOOTERS:
 if self._in_footers:
 yield "Unexpected line in footers"
@@ -146,10 +147,6 @@
 self._in_footers = True
 elif not self._in_footers:
 yield "Expected '{0}:' to be in footer".format(name)
-
-if normalized_name in BAD_FOOTERS:
-# Treat as the correct name for the rest of the rules
-normalized_name = BAD_FOOTERS[normalized_name]
 
 if normalized_name == 'bug':
 if name != 'Bug':

-- 
To view, visit https://gerrit.wikimedia.org/r/368515
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9353d07d41a2d4428189e0cc6b0b06a324c817ce
Gerrit-PatchSet: 1
Gerrit-Project: integration/commit-message-validator
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] integration/commit-message-validator[master]: [FEAT] Rewrite commit message validator

2017-07-26 Thread XZise (Code Review)

XZise has uploaded a new change for review. ( 
https://gerrit.wikimedia.org/r/367907 )

Change subject: [FEAT] Rewrite commit message validator
..

[FEAT] Rewrite commit message validator

It converts the module wide functions into a class which acts as an iterator
which can internally store information. This allows each line check to
actually know "outside information" without the need of module wide variables,
like whether it is inside the footer or whether a change id already occurred.

This also converts all of the checks into yields so that it is almost nowhere
necessary to actually provide the line number it acts on. This changes a bit
the logic but also makes sure that the line numbers are now sequential.

Change-Id: If77a0abc8c46334e6ebbf415140c12375a4870f8
---
M commit_message_validator/__init__.py
M commit_message_validator/tests/data/check_message_errors.out
2 files changed, 108 insertions(+), 119 deletions(-)


  git pull 
ssh://gerrit.wikimedia.org:29418/integration/commit-message-validator 
refs/changes/07/367907/1

diff --git a/commit_message_validator/__init__.py 
b/commit_message_validator/__init__.py
index 848d61b..175b795 100644
--- a/commit_message_validator/__init__.py
+++ b/commit_message_validator/__init__.py
@@ -81,8 +81,10 @@
 return RE_CHANGEID.match(s)
 
 
-def line_errors(lineno, line):
-"""Check a commit message line to see if it has errors.
+class CheckMessage(object):
+
+"""
+Iterator to check a commit message line for errors.
 
 Checks:
 - First line <=80 characters
@@ -94,62 +96,113 @@
 - "Change-Id:" is followed one change id ("I...")
 - No "Task: ", "Fixes: ", "Closes: " lines
 """
-# First line <=80
-if lineno == 0:
-if len(line) > 80:
-yield "First line should be <=80 characters"
-m = RE_SUBJECT_BUG_OR_TASK.match(line)
-if m:
-yield "Do not define bug in the header"
 
-# Second line blank
-elif lineno == 1:
-if line:
-yield "Second line should be empty"
+def __init__(self, lines):
+self._lines = lines
+self._first_changeid = False
+self._in_footers = False
 
-# No line >100 unless it is all a URL
-if len(line) > 100 and not RE_URL.match(line):
-yield "Line should be <=100 characters"
+def check_line(self, lineno):
+line = self._lines[lineno]
+# First line <=80
+if lineno == 0:
+if len(line) > 80:
+yield "First line should be <=80 characters"
+m = RE_SUBJECT_BUG_OR_TASK.match(line)
+if m:
+yield "Do not define bug in the header"
 
-# Look for and validate footer lines
-m = RE_FOOTER.match(line)
-if m:
-name = m.group('name')
-normalized_name = name.lower()
-ws = m.group('ws')
-value = m.group('value')
+# Second line blank
+elif lineno == 1:
+if line:
+yield "Second line should be empty"
 
-if normalized_name not in FOOTERS:
-# Meh. Not a name we care about
+# No line >100 unless it is all a URL
+elif len(line) > 100 and not RE_URL.match(line):
+yield "Line should be <=100 characters"
+
+if not line:
+if self._in_footers:
+yield "Unexpected blank line"
 return
 
-if normalized_name in BAD_FOOTERS:
-# Treat as the correct name for the rest of the rules
-normalized_name = BAD_FOOTERS[normalized_name]
+# Look for and validate footer lines
+m = RE_FOOTER.match(line)
+if m:
+name = m.group('name')
+normalized_name = name.lower()
+ws = m.group('ws')
+value = m.group('value')
 
-if normalized_name == 'bug':
-if name != 'Bug':
-yield "Use 'Bug:' not '{0}:'".format(name)
-if not is_valid_bug_id(value):
-yield "Bug: value must be a single phabricator task ID"
+if normalized_name in FOOTERS:
+if lineno > 0 and not self._lines[lineno - 1]:
+self._in_footers = True
+elif not self._in_footers:
+yield "Expected '{0}:' to be in footer".format(name)
+else:
+if self._in_footers:
+yield "Unexpected line in footers"
+else:
+# Meh. Not a name we care about
+return
 
-elif normalized_name == 'depends-on':
-if name != 'Depends-On':
-yield "Use 'Depends-On:' not '%s:'" % name
-if not is_valid_change_id(value):
-yield "Depends-On: value must be a single Gerrit change id"
+if normalized_name in BAD_FOOTERS:
+# Treat as the correct name for the rest of the

[MediaWiki-commits] [Gerrit] [IMPROV] Page repr tests: Enforce console encoding - change (pywikibot/core)

2016-01-08 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/263157

Change subject: [IMPROV] Page repr tests: Enforce console encoding
..

[IMPROV] Page repr tests: Enforce console encoding

When backporting 38589d3 to the 2.0 branch, an error occurred on the Jenkins
tests because the console encoding wasn't the expected value. Now for some
reason this is not a problem in the master branch, but in case someone uses a
custom console encoding it might fail even though it shouldn't. So this is now
enforcing the UTF-8 console encoding.

Change-Id: Ia642c34b67bf7edd0da9240a80552911b9e79918
---
M tests/page_tests.py
1 file changed, 11 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/57/263157/1

diff --git a/tests/page_tests.py b/tests/page_tests.py
index 1cc6248..48829f9 100644
--- a/tests/page_tests.py
+++ b/tests/page_tests.py
@@ -570,6 +570,17 @@
 
 """Test for Page's repr implementation."""
 
+def setUp(self):
+"""Force the console encoding to UTF-8."""
+super(TestPageRepr, self).setUp()
+self._old_encoding = config.console_encoding
+config.console_encoding = 'utf8'
+
+def tearDown(self):
+"""Restore the original console encoding."""
+config.console_encoding = self._old_encoding
+super(TestPageRepr, self).tearDown()
+
 def test_mainpage_type(self):
 u"""Test the return type of repr(Page()) is str."""
 mainpage = self.get_mainpage()

-- 
To view, visit https://gerrit.wikimedia.org/r/263157
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia642c34b67bf7edd0da9240a80552911b9e79918
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Page: Use repr-like if it can't be encoded - change (pywikibot/core)

2016-01-07 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/262923

Change subject: [FIX] Page: Use repr-like if it can't be encoded
..

[FIX] Page: Use repr-like if it can't be encoded

When the title contains characters which can't be encoded it falls back to use
an encoding which works like repr() on a unicode. It is also changing how it
behaves on Python 3 as it doesn't encode the title anymore and is thus not a
bytes anymore.

Conflicts:
pywikibot/page.py
tests/page_tests.py

Bug: T107428
Change-Id: I25dddac881891291c5a0dbe3f5dd2b1a0beedf0f
(cherry picked from commit 38589d3057847b9650b514e186b4bb3c66fad841)
---
M pywikibot/page.py
M tests/aspects.py
M tests/page_tests.py
M tests/utils.py
4 files changed, 48 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/23/262923/1

diff --git a/pywikibot/page.py b/pywikibot/page.py
index af178f0..a9c7a4e 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -273,7 +273,14 @@
 
 def __repr__(self):
 """Return a more complete string representation."""
-title = self.title().encode(config.console_encoding)
+if sys.version_info[0] > 2:
+title = repr(self.title())
+else:
+try:
+title = self.title().encode(config.console_encoding)
+except UnicodeEncodeError:
+# okay console encoding didn't work, at least try something
+title = self.title().encode('unicode_escape')
 return str('{0}({1})').format(self.__class__.__name__, title)
 
 def _cmpkey(self):
diff --git a/tests/aspects.py b/tests/aspects.py
index 4e0411c..042281f 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -907,7 +907,7 @@
 return self._mainpage
 
 mainpage = pywikibot.Page(site, site.siteinfo['mainpage'])
-if mainpage.isRedirectPage():
+if not isinstance(site, DrySite) and mainpage.isRedirectPage():
 mainpage = mainpage.getRedirectTarget()
 
 self._mainpage = mainpage
diff --git a/tests/page_tests.py b/tests/page_tests.py
index c031da4..d80f0dc 100644
--- a/tests/page_tests.py
+++ b/tests/page_tests.py
@@ -17,8 +17,8 @@
 
 from tests.aspects import (
 unittest, TestCase, DefaultSiteTestCase, SiteAttributeTestCase,
+DefaultDrySiteTestCase,
 )
-from tests.utils import expected_failure_if
 
 if sys.version_info[0] > 2:
 basestring = (str, )
@@ -507,11 +507,20 @@
 self.assertTrue(page_copy.isRedirectPage())
 
 
-class TestPageRepr(DefaultSiteTestCase):
+class TestPageBaseUnicode(DefaultDrySiteTestCase):
 
-"""Test Page representation."""
+"""Base class for tests requring a page using a unicode title."""
 
-cached = True
+@classmethod
+def setUpClass(cls):
+"""Initialize page instance."""
+super(TestPageBaseUnicode, cls).setUpClass()
+cls.page = pywikibot.Page(cls.site, 'Ō')
+
+
+class TestPageRepr(TestPageBaseUnicode):
+
+"""Test for Page's repr implementation."""
 
 def test_mainpage_type(self):
 u"""Test the return type of repr(Page()) is str."""
@@ -523,7 +532,7 @@
 page = pywikibot.Page(self.get_site(), u'Ō')
 self.assertIsInstance(repr(page), str)
 
-@expected_failure_if(sys.version_info[0] > 2)
+@unittest.skipIf(sys.version_info[0] > 2, 'Python 2 specific test')
 def test_unicode_value(self):
 """Test repr(Page(u'')) is represented simply as utf8."""
 page = pywikibot.Page(self.get_site(), u'Ō')
@@ -539,10 +548,30 @@
 @unittest.skipIf(sys.version_info[0] < 3, 'Python 3+ specific test')
 def test_unicode_value_py3(self):
 """Test to capture actual Python 3 result pre unicode_literals."""
-page = pywikibot.Page(self.get_site(), u'Ō')
-self.assertEqual(repr(page), "Page(b'\\xc5\\x8c')")
-self.assertEqual(u'%r' % page, "Page(b'\\xc5\\x8c')")
-self.assertEqual(u'{0!r}'.format(page), "Page(b'\\xc5\\x8c')")
+self.assertEqual(repr(self.page), "Page('Ō')")
+self.assertEqual('%r' % self.page, "Page('Ō')")
+self.assertEqual('{0!r}'.format(self.page), "Page('Ō')")
+
+
+class TestPageReprASCII(TestPageBaseUnicode):
+
+"""Test for Page's repr implementation when using ASCII encoding."""
+
+def setUp(self):
+"""Patch the current console encoding to ASCII."""
+super(TestPageReprASCII, self).setUp()
+self._old_encoding = config.console_encoding
+config.console_encoding = 'ascii'
+
+def tearDown(self):
+"""Restore the original console encoding."""
+config.console_encoding = self._old_encoding
+super(TestPageReprASCII, self).tearDown()
+
+@unittest.skipIf(sys.version_info[0] > 2, 'Python 2 specific test')
+def test_incapable_encoding(self):
+"""Test that repr still works even if the console

[MediaWiki-commits] [Gerrit] [FIX] Support changed _vformat signature of 3.5.1 - change (pywikibot/core)

2015-12-16 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/259549

Change subject: [FIX] Support changed _vformat signature of 3.5.1
..

[FIX] Support changed _vformat signature of 3.5.1

With Python 3.5.1 the signature of `Formatter._vformat` changed. This is
supporting this change and now returns the tuple if it had returned a tuple
first.

This is not deactivating the custom `_vformat` method in Python 3 because the
patch introduced that change might be backported to Python 2.7 (see also
https://bugs.python.org/issue25034).

Bug: T121684
Change-Id: I0d1e4f062882ed03dc7bb2040984a67970df6c52
---
M pywikibot/tools/formatter.py
1 file changed, 12 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/49/259549/1

diff --git a/pywikibot/tools/formatter.py b/pywikibot/tools/formatter.py
index ee34057..a571ee4 100644
--- a/pywikibot/tools/formatter.py
+++ b/pywikibot/tools/formatter.py
@@ -123,6 +123,18 @@
 @rtype: unicode
 """
 result = super(_ColorFormatter, self)._vformat(*args, **kwargs)
+if isinstance(result, tuple):
+additional_params = result[1:]
+result = result[0]
+else:
+additional_params = tuple()
+result = self._convert_bytes(result)
+if additional_params:
+result = (result, ) + additional_params
+return result
+
+def _convert_bytes(self, result):
+"""Convert everything into unicode."""
 if PY2 and isinstance(result, str):
 assert result == b''
 result = ''  # This is changing it into a unicode

-- 
To view, visit https://gerrit.wikimedia.org/r/259549
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0d1e4f062882ed03dc7bb2040984a67970df6c52
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Update domain for SSL expiry tests - change (pywikibot/core)

2015-12-16 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/259559

Change subject: [FIX] Update domain for SSL expiry tests
..

[FIX] Update domain for SSL expiry tests

The domain has changed from testssl-expire.disig.sk to
testssl-expire-r2i2.disig.sk.

Change-Id: I9d8543fa1ee36684807ca82e3921a174e86c7c2b
---
M tests/http_tests.py
1 file changed, 4 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/59/259559/1

diff --git a/tests/http_tests.py b/tests/http_tests.py
index 03a5092..4922a81 100644
--- a/tests/http_tests.py
+++ b/tests/http_tests.py
@@ -124,17 +124,17 @@
 
 """HTTPS certificate test."""
 
-hostname = 'testssl-expire.disig.sk'
+hostname = 'testssl-expire-r2i2.disig.sk'
 
 def test_https_cert_error(self):
 """Test if http.fetch respects disable_ssl_certificate_validation."""
 self.assertRaises(pywikibot.FatalServerError,
   http.fetch,
-  uri='https://testssl-expire.disig.sk/index.en.html')
+  
uri='https://testssl-expire-r2i2.disig.sk/index.en.html')
 
 with warnings.catch_warnings(record=True) as warning_log:
 response = http.fetch(
-uri='https://testssl-expire.disig.sk/index.en.html',
+uri='https://testssl-expire-r2i2.disig.sk/index.en.html',
 disable_ssl_certificate_validation=True)
 r = response.content
 self.assertIsInstance(r, unicode)
@@ -144,7 +144,7 @@
 http.session.close()  # but first clear the connection
 self.assertRaises(pywikibot.FatalServerError,
   http.fetch,
-  uri='https://testssl-expire.disig.sk/index.en.html')
+  
uri='https://testssl-expire-r2i2.disig.sk/index.en.html')
 
 # Verify that the warning occurred
 self.assertEqual(len(warning_log), 1)

-- 
To view, visit https://gerrit.wikimedia.org/r/259559
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9d8543fa1ee36684807ca82e3921a174e86c7c2b
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Remove unnecessary variable definitions - change (pywikibot/core)

2015-12-02 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/256411

Change subject: [IMPROV] Remove unnecessary variable definitions
..

[IMPROV] Remove unnecessary variable definitions

Defining a function locale variable directly before a `return` statement which
is not using that variable is unnecessary.

Change-Id: Idf623fad2498a0395b8b40a6bc2b78bba15fc412
---
M scripts/checkimages.py
1 file changed, 0 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/11/256411/1

diff --git a/scripts/checkimages.py b/scripts/checkimages.py
index 0e752e5..762f6d6 100755
--- a/scripts/checkimages.py
+++ b/scripts/checkimages.py
@@ -1532,7 +1532,6 @@
 # Here begins the check block.
 if brackets and license_found:
 # It works also without this... but i want only to be sure ^^
-brackets = False
 return True
 elif delete:
 pywikibot.output(u"%s is not a file!" % self.imageName)
@@ -1542,7 +1541,6 @@
 notification = din % self.imageName
 head = dih
 self.report(canctext, self.imageName, notification, head)
-delete = False
 return True
 elif self.imageCheckText in nothing:
 pywikibot.output(

-- 
To view, visit https://gerrit.wikimedia.org/r/256411
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Idf623fad2498a0395b8b40a6bc2b78bba15fc412
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Simplify findmarker usage - change (pywikibot/core)

2015-11-19 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/254139

Change subject: [IMPROV] Simplify findmarker usage
..

[IMPROV] Simplify findmarker usage

In most cases the marker got by `findmarker` consists of one character and it
starts using that twice and adds that one character each time. And in theory
any other way is not necessary to determine a marker. So this is replacing
`findmarker` by `generate_marker` which only accepts one character as a
parameter.

Change-Id: I5c253dffbf61ddc205eb4429c60aab9314c149ef
---
M pywikibot/textlib.py
1 file changed, 23 insertions(+), 12 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/39/254139/1

diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index f5c2888..95dd945 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -428,16 +428,27 @@
 For the tags parameter, see L{removeDisabledParts}.
 """
 # Find a marker that is not already in the text.
-marker = findmarker(text)
+marker = generate_marker(text)
 text = text[:index] + marker + text[index:]
 text = removeDisabledParts(text, tags)
 return (marker not in text)
 
 
+@deprecated('generate_marker')
 def findmarker(text, startwith=u'@@', append=None):
 """Find a string which is not part of text."""
 if not append:
 append = u'@'
+return _generate_marker(text, startwith, append)
+
+
+def generate_marker(text, base='@'):
+"""Find a string which is not part of text."""
+assert len(base) == 1
+return _generate_marker(text, base * 2, base)
+
+
+def _generate_marker(text, startwith, append):
 mymarker = startwith
 while mymarker in text:
 mymarker += append
@@ -831,7 +842,7 @@
 
 """
 if separator:
-mymarker = findmarker(text, u'@L@')
+mymarker = generate_marker(text, '%')
 newtext = removeLanguageLinks(text, site, mymarker)
 mymarker = expandmarker(newtext, mymarker, separator)
 return newtext.replace(mymarker, marker)
@@ -848,7 +859,7 @@
 function).
 """
 # Find a marker that is not already in the text.
-marker = findmarker(oldtext)
+marker = generate_marker(oldtext)
 if site is None:
 site = pywikibot.Site()
 separator = site.family.interwiki_text_separator
@@ -1065,7 +1076,7 @@
 if site is None:
 site = pywikibot.Site()
 if separator:
-mymarker = findmarker(text, u'@C@')
+mymarker = generate_marker(text, '§')
 newtext = removeCategoryLinks(text, site, mymarker)
 mymarker = expandmarker(newtext, mymarker, separator)
 return newtext.replace(mymarker, marker)
@@ -1132,7 +1143,7 @@
 category(s) given will be added (and so they won't replace anything).
 """
 # Find a marker that is not already in the text.
-marker = findmarker(oldtext)
+marker = generate_marker(oldtext)
 if site is None:
 site = pywikibot.Site()
 if site.sitename == 'wikipedia:de' and '{{Personendaten' in oldtext:
@@ -1386,16 +1397,16 @@
 thistxt = text
 
 # marker for inside templates or parameters
-marker1 = findmarker(thistxt)
+marker1 = generate_marker(thistxt)
 
 # marker for links
-marker2 = findmarker(thistxt, u'##', u'#')
+marker2 = generate_marker(thistxt, '#')
 
 # marker for math
-marker3 = findmarker(thistxt, u'%%', u'%')
+marker3 = generate_marker(thistxt, '%')
 
 # marker for value parameter
-marker4 = findmarker(thistxt, u'§§', u'§')
+marker4 = generate_marker(thistxt, '§')
 
 result = []
 Rmath = re.compile(r'[^<]+')
@@ -1748,10 +1759,10 @@
 self.tzinfo = tzoneFixedOffset(self.site.siteinfo['timeoffset'],
self.site.siteinfo['timezone'])
 
-@deprecated('module function')
+@deprecated('generate_marker')
 def findmarker(self, text, base=u'@@', delta='@'):
 """Find a string which is not part of text."""
-return findmarker(text, base, delta)
+return _generate_marker(text, base, delta)
 
 def fix_digits(self, line):
 """Make non-latin digits like Persian to latin to parse."""
@@ -1772,7 +1783,7 @@
 cnt += 1
 
 if m:
-marker = findmarker(txt)
+marker = generate_marker(txt)
 # month and day format might be identical (e.g. see bug 69315),
 # avoid to wipe out day, after month is matched.
 # replace all matches but the last two

-- 
To view, visit https://gerrit.wikimedia.org/r/254139
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5c253dffbf61ddc205eb4429c60aab9314c149ef
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___

[MediaWiki-commits] [Gerrit] [IMPROV] TimeStripper: Deprecate duplicate findmarker - change (pywikibot/core)

2015-11-18 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/253965

Change subject: [IMPROV] TimeStripper: Deprecate duplicate findmarker
..

[IMPROV] TimeStripper: Deprecate duplicate findmarker

The `findmarker` method in `TimeStripper` works exactly the same as the
function in the module. This is deprecating the method in the class in favour
of the module function.

This is also introducing a new super class in the corresponding tests which
will take care of creating the `TimeStripper` instance.

Change-Id: Ia349e78e68d94fc6c6ceab75d8b679bbee48b2b7
---
M pywikibot/textlib.py
M tests/timestripper_tests.py
2 files changed, 32 insertions(+), 24 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/65/253965/1

diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index 54d20eb..f5c2888 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -39,6 +39,7 @@
 from pywikibot.exceptions import InvalidTitle
 from pywikibot.family import Family
 from pywikibot.tools import (
+deprecated,
 DeprecatedRegex,
 OrderedDict,
 UnicodeType,
@@ -1747,11 +1748,10 @@
 self.tzinfo = tzoneFixedOffset(self.site.siteinfo['timeoffset'],
self.site.siteinfo['timezone'])
 
+@deprecated('module function')
 def findmarker(self, text, base=u'@@', delta='@'):
 """Find a string which is not part of text."""
-while base in text:
-base += delta
-return base
+return findmarker(text, base, delta)
 
 def fix_digits(self, line):
 """Make non-latin digits like Persian to latin to parse."""
@@ -1772,7 +1772,7 @@
 cnt += 1
 
 if m:
-marker = self.findmarker(txt)
+marker = findmarker(txt)
 # month and day format might be identical (e.g. see bug 69315),
 # avoid to wipe out day, after month is matched.
 # replace all matches but the last two
diff --git a/tests/timestripper_tests.py b/tests/timestripper_tests.py
index 61a20ab..4c5a9fc 100644
--- a/tests/timestripper_tests.py
+++ b/tests/timestripper_tests.py
@@ -13,28 +13,45 @@
 
 from pywikibot.textlib import TimeStripper, tzoneFixedOffset
 
-from tests.aspects import unittest, TestCase
+from tests.aspects import (
+unittest,
+TestCase,
+DefaultSiteTestCase,
+DeprecationTestCase,
+)
 
 
-class TestTimeStripperWithNoDigitsAsMonths(TestCase):
+class TestTimeStripperCase(TestCase):
 
-"""Test cases for TimeStripper methods."""
-
-family = 'wikipedia'
-code = 'fr'
+"""Basic class to test the TimeStripper class."""
 
 cached = True
 
 def setUp(self):
 """Set up test cases."""
-super(TestTimeStripperWithNoDigitsAsMonths, self).setUp()
+super(TestTimeStripperCase, self).setUp()
 self.ts = TimeStripper(self.get_site())
+
+
+class DeprecatedTestTimeStripperCase(TestTimeStripperCase, DeprecationTestCase,
+ DefaultSiteTestCase):
+
+"""Test deprecated parts of the TimeStripper class."""
 
 def test_findmarker(self):
 """Test that string which is not part of text is found."""
 txt = u'this is a string with a maker is already present'
 self.assertEqual(self.ts.findmarker(txt, base=u'@@', delta='@@'),
  '@@')
+self.assertOneDeprecation()
+
+
+class TestTimeStripperWithNoDigitsAsMonths(TestTimeStripperCase):
+
+"""Test cases for TimeStripper methods."""
+
+family = 'wikipedia'
+code = 'fr'
 
 def test_last_match_and_replace(self):
 """Test that pattern matches and removes items correctly."""
@@ -88,19 +105,12 @@
 self.assertEqual(self.ts.timestripper(txtHourOutOfRange), None)
 
 
-class TestTimeStripperWithDigitsAsMonths(TestCase):
+class TestTimeStripperWithDigitsAsMonths(TestTimeStripperCase):
 
 """Test cases for TimeStripper methods."""
 
 family = 'wikipedia'
 code = 'cs'
-
-cached = True
-
-def setUp(self):
-"""Setup a timestripper for the configured site."""
-super(TestTimeStripperWithDigitsAsMonths, self).setUp()
-self.ts = TimeStripper(self.get_site())
 
 def test_last_match_and_replace(self):
 """Test that pattern matches and removes items correctly."""
@@ -236,7 +246,7 @@
 self.assertEqual(self.ts.timestripper(txtNoMatch), None)
 
 
-class TestTimeStripperDoNotArchiveUntil(TestCase):
+class TestTimeStripperDoNotArchiveUntil(TestTimeStripperCase):
 
 """Test cases for Do Not Archive Until templates.
 
@@ -247,8 +257,6 @@
 family = 'wikisource'
 code = 'en'
 
-cached = True
-
 username = '[[User:DoNotArchiveUntil]]'
 date = '06:57 06 June 2015 (UTC)'
 user_and_date = username + ' ' + date
@@ -256,7 +264,7 @@
 
 def test_timestripper_match(self):
 """Test that

[MediaWiki-commits] [Gerrit] [FIX] tools_tests: Prevent overwriting name parameter - change (pywikibot/core)

2015-11-04 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/250940

Change subject: [FIX] tools_tests: Prevent overwriting name parameter
..

[FIX] tools_tests: Prevent overwriting name parameter

The `name` parameter for the `__new__` method was accidentally overwritten when
it created the tests with baf31f1b.

Change-Id: I9473d7132ca031956edc27db7f23f69941ed3c89
---
M tests/tools_tests.py
1 file changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/40/250940/1

diff --git a/tests/tools_tests.py b/tests/tools_tests.py
index c58e33f..2d0935e 100644
--- a/tests/tools_tests.py
+++ b/tests/tools_tests.py
@@ -506,9 +506,9 @@
 self.assertNoDeprecation()
 return test_method
 
-for name, tested_method in list(dct.items()):
-if name.startswith('_method_test_'):
-suffix = name[len('_method_test_'):]
+for attr, tested_method in list(dct.items()):
+if attr.startswith('_method_test_'):
+suffix = attr[len('_method_test_'):]
 cls.add_method(dct, 'test_method_' + suffix,
create_test(tested_method),
doc_suffix='on {0}'.format(suffix))

-- 
To view, visit https://gerrit.wikimedia.org/r/250940
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9473d7132ca031956edc27db7f23f69941ed3c89
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] cosmetic_changes: Allow missing HTTPS hostname - change (pywikibot/core)

2015-10-26 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/248875

Change subject: [FIX] cosmetic_changes: Allow missing HTTPS hostname
..

[FIX] cosmetic_changes: Allow missing HTTPS hostname

If there is not HTTPS hostname it can't compare that hostname with the HTTP
hostname so it should skip it. The loop below creating a list for each separate
protocol is already supporting the HTTPS hostname to be `None`.

Bug: T116585
Change-Id: If37f1ec9fcdf653ef3f4dfa2712da2f55846227f
---
M pywikibot/cosmetic_changes.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/75/248875/1

diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py
index c27f9bc..8aabc09 100755
--- a/pywikibot/cosmetic_changes.py
+++ b/pywikibot/cosmetic_changes.py
@@ -712,7 +712,7 @@
 https_url = self.site.base_url(suffix, 'https')
 # compare strings without the protocol, if they are empty support
 # also no prefix (//en.wikipedia.org/…)
-if http_url[4:] == https_url[5:]:
+if https_url is not None and http_url[4:] == https_url[5:]:
 urls = ['(?:https?:)?' + re.escape(http_url[5:])]
 else:
 urls = [re.escape(url) for url in (http_url, https_url)

-- 
To view, visit https://gerrit.wikimedia.org/r/248875
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If37f1ec9fcdf653ef3f4dfa2712da2f55846227f
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Yield interwiki URLs separately - change (pywikibot/core)

2015-10-20 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247557

Change subject: [IMPROV] Yield interwiki URLs separately
..

[IMPROV] Yield interwiki URLs separately

Instead of returning one list with all URLs it can yield one URL at a time.
This way it's possible to only query the article path if nothing else matched
first.

Change-Id: Idd20f0757856299d2ae21608bb29d12c3e4cae11
---
M pywikibot/site.py
1 file changed, 5 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/57/247557/1

diff --git a/pywikibot/site.py b/pywikibot/site.py
index 16d3e99..1c66d1e 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -918,11 +918,11 @@
 if self.namespaces.lookup_normalized_name(lang) is None]
 
 def _interwiki_urls(self):
-site_paths = [self.path()] * 3
-site_paths[1] += '/'
-site_paths[2] += '?title='
-site_paths += [self.article_path]
-return site_paths
+base_path = self.path()
+yield base_path
+yield base_path + '/'
+yield base_path + '?title='
+yield self.article_path
 
 def interwiki(self, prefix):
 """

-- 
To view, visit https://gerrit.wikimedia.org/r/247557
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Idd20f0757856299d2ae21608bb29d12c3e4cae11
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] site: Dynamically determine Wikibase repo - change (pywikibot/core)

2015-10-20 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247555

Change subject: [FEAT] site: Dynamically determine Wikibase repo
..

[FEAT] site: Dynamically determine Wikibase repo

Instead of statically defining the Wikibase repository in the family, it can
query the API to determine the repository without user input.

Bug: T85331
Change-Id: I5568d848a369c8af36a40c193e2409071e23a5fb
---
M pywikibot/site.py
1 file changed, 16 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/55/247555/1

diff --git a/pywikibot/site.py b/pywikibot/site.py
index 16d3e99..f67da05 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -2654,10 +2654,22 @@
 
 def data_repository(self):
 """Return Site object for data repository e.g. Wikidata."""
-code, fam = self.shared_data_repository()
-if bool(code or fam):
-return pywikibot.Site(code, fam, self.username(),
-  interface="DataSite")
+def handle_warning(mod, warning):
+return (mod == 'query' and
+warning == "Unrecognized value for parameter 'meta': "
+   "wikibase")
+
+req = self._simple_request(action='query', meta='wikibase')
+req._warning_handler = handle_warning
+data = req.submit()
+if 'query' in data and 'wikibase' in data['query']:
+data = data['query']['wikibase']['repo']['url']
+return pywikibot.Site(
+url=data['base'] + data['scriptpath'] + '/index.php',
+user=self.username(), interface='DataSite')
+else:
+assert 'warnings' in data
+return None
 
 def is_image_repository(self):
 """Return True if Site object is the image repository."""

-- 
To view, visit https://gerrit.wikimedia.org/r/247555
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5568d848a369c8af36a40c193e2409071e23a5fb
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] cache: Separate commands - change (pywikibot/core)

2015-10-19 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247482

Change subject: [FEAT] cache: Separate commands
..

[FEAT] cache: Separate commands

Instead of having everything in one command and to play with `and` to only
execute part of the command, this is providing three separate commands to
filter for entries, output something depending on them and to do an action
afterwards.

The filter command only needs to return a boolean for an entry. The output
command can either return None to suppress the default output, similar to
before. If it returns anything else it'll print that result. And at the moment
only the delete action is supported and that as an option and not using code
like the other two.

Additionally it supports to only define a function name and it'll call it
automatically with the entry as the first positional parameter.

It adds two utility output functions to easier see what each cache entry is
actually doing.

Change-Id: I7b8fc283555a6d35719e47d28ff1becfbd07baf7
---
M scripts/maintenance/cache.py
1 file changed, 97 insertions(+), 24 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/82/247482/1

diff --git a/scripts/maintenance/cache.py b/scripts/maintenance/cache.py
index 8aa15b0..14188a1 100755
--- a/scripts/maintenance/cache.py
+++ b/scripts/maintenance/cache.py
@@ -5,29 +5,43 @@
 
 Syntax:
 
-python pwb.py cache [-password] [-delete] [-c "..."] [dir ...]
+python pwb.py cache [-password] [-delete] [-c "..."] [-o "..."] [dir ...]
 
 If no directory are specified, it will detect the API caches.
 
 If no command is specified, it will print the filename of all entries.
 If only -delete is specified, it will delete all entries.
 
-The option '-c' must be followed by a command in python syntax.
+-delete   Delete each command filtered. If that option is set the
+  default output will be nothing.
+
+-cFilter command in python syntax. It must evaulate to True to
+  output anything.
+
+-oOutput command which is output when the filter evaluated to
+  True. If it returns None it won't output anything.
 
 Example commands:
   Print the filename of any entry with 'wikidata' in the key:
 
-entry if "wikidata" in entry._uniquedescriptionstr() else None
+-c "wikidata" in entry._uniquedescriptionstr()
 
   Customised output if the site code is 'ar':
 
-entry.site.code == "ar" and print("%s" % entry._uniquedescriptionstr())
+-c entry.site.code == "ar"
+-o uniquedesc(entry)
 
   Or the state of the login
-entry.site._loginstatus == LoginStatus.NOT_ATTEMPTED and \
-print("%s" % entry._uniquedescriptionstr())
 
-  These functions can be used as a command:
+-c entry.site._loginstatus == LoginStatus.NOT_ATTEMPTED
+-o uniquedesc(entry)
+
+  If the function only uses one parameter for the entry it can be omitted:
+
+-c has_password
+-o uniquedesc
+
+Available filter commands:
 has_password(entry)
 is_logout(entry)
 empty_response(entry)
@@ -39,9 +53,13 @@
   There are helper functions which can be part of a command:
 older_than(entry, interval)
 newer_than(entry, interval)
+
+Available output commands:
+
+uniquedesc(entry)
 """
 #
-# (C) Pywikibot team, 2014
+# (C) Pywikibot team, 2014-2015
 #
 # Distributed under the terms of the MIT license.
 #
@@ -182,7 +200,8 @@
 os.remove(self._cachefile_path())
 
 
-def process_entries(cache_path, func, use_accesstime=None):
+def process_entries(cache_path, func, use_accesstime=None, output_func=None,
+action_func=None):
 """
 Check the contents of the cache.
 
@@ -251,8 +270,30 @@
 pywikibot.exception(e, tb=True)
 continue
 
-func(entry)
+if func(entry):
+if output_func:
+output = output_func(entry)
+if output is not None:
+pywikibot.output(output)
+if action_func:
+action_func(entry)
 
+
+def _parse_command(command, name):
+obj = globals().get(command)
+if callable(obj):
+return obj
+else:
+try:
+return eval('lambda entry: ' + command)
+except:
+pywikibot.exception()
+pywikibot.error('Can not compile {0} command: '
+'{1}'.format(name, command))
+return None
+
+
+# Filter commands
 
 def has_password(entry):
 """Entry has a password in the entry."""
@@ -306,15 +347,39 @@
 return entry
 
 
+# Output commands
+
+def uniquedesc(entry):
+"""Return the unique description string."""
+return entry._uniquedescriptionstr()
+
+
+def parameters(entry):
+"""Return a pretty formatted parameters list."""
+lines = ''
+for key, items in sorted(entry._params.items()):
+lines +=

[MediaWiki-commits] [Gerrit] [FIX] Prevent string comparison for Coordinate - change (pywikibot/core)

2015-10-17 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247042

Change subject: [FIX] Prevent string comparison for Coordinate
..

[FIX] Prevent string comparison for Coordinate

When it tries to compare a string to `Coordinate` it previously parsed the
string and returned True if the string matched the coordinate. Otherwise it
just continued to work on it and compared the object with the string directly
which never evaluated to True, so it always returned False.

But with 4f4770f3 `Coordinate` implements `__eq__` and now that method tries to
compare a string to a `Coordinate` and that raises a `AttributeError`. So
instead of just returning True and falling back to the default comparison
otherwise it directly returns False if the comparison with the parsed string
failed.

Change-Id: Ibe322383a319c57fd20db43758f87864550ea0e4
---
M pywikibot/page.py
1 file changed, 2 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/42/247042/1

diff --git a/pywikibot/page.py b/pywikibot/page.py
index a69f09a..652773e 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -4428,9 +4428,8 @@
 except TypeError:
 pass
 
-if (abs(self.target.lat - coord_args[0]) <= precision and
-abs(self.target.lon - coord_args[1]) <= precision):
-return True
+return (abs(self.target.lat - coord_args[0]) <= precision and
+abs(self.target.lon - coord_args[1]) <= precision)
 
 if self.target == value:
 return True

-- 
To view, visit https://gerrit.wikimedia.org/r/247042
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ibe322383a319c57fd20db43758f87864550ea0e4
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] login: Inform about unsupported -pass - change (pywikibot/core)

2015-10-17 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247039

Change subject: [IMPROV] login: Inform about unsupported -pass
..

[IMPROV] login: Inform about unsupported -pass

The `-pass` parameter is not supported yet so inform the user that defining one
is not actually using that password.

Bug: T102477
Change-Id: I072481cbeb3b089d796a739d2af348f9a26d2dca
---
M scripts/login.py
1 file changed, 4 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/39/247039/1

diff --git a/scripts/login.py b/scripts/login.py
index a88dff3..f156817 100755
--- a/scripts/login.py
+++ b/scripts/login.py
@@ -148,6 +148,10 @@
 pywikibot.bot.suggest_help(unknown_parameters=unknown_args)
 return False
 
+if password is not None:
+pywikibot.warning('The -pass argument is not implemented yet. See: '
+  'https://phabricator.wikimedia.org/T102477')
+
 if logall:
 if sysop and not oauth:
 namedict = config.sysopnames

-- 
To view, visit https://gerrit.wikimedia.org/r/247039
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I072481cbeb3b089d796a739d2af348f9a26d2dca
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] Allow tests without tox - change (integration/jenkins)

2015-10-17 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247046

Change subject: [FEAT] Allow tests without tox
..

[FEAT] Allow tests without tox

To test only one module, class or method it's possible to use a syntax like:

python -m unittest -v tests.test_commit-message-validator

But that does not work if the test is importing the tested module without using
an absolute path.

Change-Id: I9f4fb85928962611ee0dfd4537593c1cf1691458
---
M tests/test_commit-message-validator.py
M tests/test_mw-create-composer-local.py
2 files changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/46/247046/1

diff --git a/tests/test_commit-message-validator.py 
b/tests/test_commit-message-validator.py
index 6417cba..7cca2fc 100644
--- a/tests/test_commit-message-validator.py
+++ b/tests/test_commit-message-validator.py
@@ -12,7 +12,7 @@
 
 cmv = imp.load_source(
 'cmv',
-os.path.dirname(os.path.dirname(__file__)) +
+os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +
 '/tools/commit-message-validator.py')
 
 
diff --git a/tests/test_mw-create-composer-local.py 
b/tests/test_mw-create-composer-local.py
index dbce092..62ffad4 100644
--- a/tests/test_mw-create-composer-local.py
+++ b/tests/test_mw-create-composer-local.py
@@ -4,7 +4,7 @@
 import os
 import tempfile
 import unittest
-path = os.path.dirname(os.path.dirname(__file__)) \
+path = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) \
 + '/bin/mw-create-composer-local.py'
 mw_create_composer_local = imp.load_source('mw-create-composer-local', path)
 

-- 
To view, visit https://gerrit.wikimedia.org/r/247046
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9f4fb85928962611ee0dfd4537593c1cf1691458
Gerrit-PatchSet: 1
Gerrit-Project: integration/jenkins
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] Verify phabricator bug ids - change (integration/jenkins)

2015-10-17 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247048

Change subject: [FEAT] Verify phabricator bug ids
..

[FEAT] Verify phabricator bug ids

This requires that the bug ids are always phabricator ids (meaning that they
use the format `T\d+`). This is applied to all `Bug:` lines and similar
sounding prefixes.

It also does asserts if the commit message header does not start with a bug id.

Change-Id: Ib94bf559e8f7b14e93622e96a8af8149f50a08d3
---
M tests/test_commit-message-validator/check_message_errors.out
M tools/commit-message-validator.py
2 files changed, 13 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/48/247048/1

diff --git a/tests/test_commit-message-validator/check_message_errors.out 
b/tests/test_commit-message-validator/check_message_errors.out
index 43d414d..a54cc06 100644
--- a/tests/test_commit-message-validator/check_message_errors.out
+++ b/tests/test_commit-message-validator/check_message_errors.out
@@ -3,11 +3,15 @@
 Line 3: Line should be <=100 characters
 Line 4: Use 'Bug: ' not 'Fixes:'
 Line 4: Expected one space after 'Bug:'
+Line 4: The bug ID is not a phabricator task: foo
 Line 5: Use 'Bug: ' not 'Closes:'
 Line 5: Expected one space after 'Bug:'
+Line 5: The bug ID is not a phabricator task: bar
 Line 6: Use 'Bug: ' not 'Task:'
 Line 7: Expected 'Bug:' not 'BUG:'
 Line 7: Expected one space after 'Bug:'
+Line 7: The bug ID must be a phabricator task: 13
+Line 8: The bug ID is not a phabricator task: T11 T12
 Line 8: Expected blank line before Bug:
 Line 9: Unexpected blank line after Bug:
 Line 12: Unexpected blank line after Bug:
diff --git a/tools/commit-message-validator.py 
b/tools/commit-message-validator.py
index c0f4ba9..790e9d6 100755
--- a/tools/commit-message-validator.py
+++ b/tools/commit-message-validator.py
@@ -31,6 +31,9 @@
 if lineno == 0:
 if len(line) > 80:
 yield "First line should be <=80 characters"
+m = re.match(r'^T?\d+', line)
+if m:
+yield "Do not define bug in the header"
 
 # Second line blank
 elif lineno == 1:
@@ -56,6 +59,12 @@
 if m.group(2) != ' ':
 yield "Expected one space after 'Bug:'"
 
+bug_id = m.group(3).strip()
+if bug_id.isdigit():
+yield "The bug ID must be a phabricator task: {0}".format(bug_id)
+elif not bug_id.startswith('T') or not bug_id[1:].isdigit():
+yield "The bug ID is not a phabricator task: {0}".format(bug_id)
+
 
 def check_message(lines):
 """Check a commit message to see if it has errors.

-- 
To view, visit https://gerrit.wikimedia.org/r/247048
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib94bf559e8f7b14e93622e96a8af8149f50a08d3
Gerrit-PatchSet: 1
Gerrit-Project: integration/jenkins
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] Report multiple errors on each line - change (integration/jenkins)

2015-10-17 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247047

Change subject: [FEAT] Report multiple errors on each line
..

[FEAT] Report multiple errors on each line

Instead of just reporting the first error it should report every error on each
line so that the user can fix them all at one instead of step by step.

Similar to the verification after `Bug:` it also verifies for similar sounding
names if they follow the same style guide.

Change-Id: Id27070a5db98959523fba15c65964c2384c8a79b
---
M tests/test_commit-message-validator/check_message_errors.out
M tools/commit-message-validator.py
2 files changed, 26 insertions(+), 28 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/47/247047/1

diff --git a/tests/test_commit-message-validator/check_message_errors.out 
b/tests/test_commit-message-validator/check_message_errors.out
index 1fa40bd..43d414d 100644
--- a/tests/test_commit-message-validator/check_message_errors.out
+++ b/tests/test_commit-message-validator/check_message_errors.out
@@ -2,10 +2,12 @@
 Line 2: Second line should be empty
 Line 3: Line should be <=100 characters
 Line 4: Use 'Bug: ' not 'Fixes:'
+Line 4: Expected one space after 'Bug:'
 Line 5: Use 'Bug: ' not 'Closes:'
+Line 5: Expected one space after 'Bug:'
 Line 6: Use 'Bug: ' not 'Task:'
 Line 7: Expected 'Bug:' not 'BUG:'
-Line 8: Each Bug: should list exactly one task
+Line 7: Expected one space after 'Bug:'
 Line 8: Expected blank line before Bug:
 Line 9: Unexpected blank line after Bug:
 Line 12: Unexpected blank line after Bug:
diff --git a/tools/commit-message-validator.py 
b/tools/commit-message-validator.py
index 44e3a28..c0f4ba9 100755
--- a/tools/commit-message-validator.py
+++ b/tools/commit-message-validator.py
@@ -13,7 +13,7 @@
 import subprocess
 
 
-def line_has_errors(lineno, line):
+def line_errors(lineno, line):
 """Check a commit message line to see if it has errors.
 
 Checks:
@@ -28,43 +28,40 @@
 rline = lineno + 1
 
 # First line <=80
-if lineno == 0 and len(line) > 80:
-return "Line %d: First line should be <=80 characters" % rline
+if lineno == 0:
+if len(line) > 80:
+yield "First line should be <=80 characters"
 
 # Second line blank
-if lineno == 1 and line:
-return "Line %d: Second line should be empty" % rline
+elif lineno == 1:
+if line:
+yield "Second line should be empty"
 
 # No line >100
-if len(line) > 100:
-return "Line %d: Line should be <=100 characters" % rline
+elif len(line) > 100:
+yield "Line should be <=100 characters"
 
-m = re.match(r'^(bug:)(.)', line, re.IGNORECASE)
+m = re.match(r'^(bug|closes|fixes|task):(\W)*(.*)', line, re.IGNORECASE)
 if m:
-if m.group(1) != 'Bug:':
-return "Line %d: Expected 'Bug:' not '%s'" % (rline, m.group(1))
+if lineno == 0:
+yield "Do not define bug in the header"
+
+if m.group(1).lower() == 'bug':
+if m.group(1) != 'Bug':
+yield "Expected 'Bug:' not '%s:'" % m.group(1)
+else:
+# No "Task: ", "Fixes: ", "Closes: " lines
+yield "Use 'Bug: ' not '%s:'" % m.group(1)
 
 if m.group(2) != ' ':
-return "Line %d: Expected space after 'Bug:'" % rline
-
-# Exactly one task id on each Bug: line
-m = re.match(r'^Bug: \w+$', line)
-if not m:
-return "Line %d: Each Bug: should list exactly one task" % rline
-
-# No "Task: ", "Fixes: ", "Closes: " lines
-m = re.match(r'^(closes|fixes|task):', line, re.IGNORECASE)
-if m:
-return "Line %d: Use 'Bug: ' not '%s'" % (rline, m.group(0))
-
-return False
+yield "Expected one space after 'Bug:'"
 
 
 def check_message(lines):
 """Check a commit message to see if it has errors.
 
 Checks:
-- All lines ok as checked by line_has_errors()
+- All lines ok as checked by line_errors()
 - For any "^Bug: " line, next line is not blank
 - For any "^Bug: " line, prior line is another Bug: line or empty
 - Exactly one "Change-Id: " line per commit
@@ -80,9 +77,8 @@
 last_bug = False
 for lineno, line in enumerate(lines):
 rline = lineno + 1
-e = line_has_errors(lineno, line)
-if e:
-errors.append(e)
+errors.extend('Line {0}: {1}'.format(rline, e)
+  for e in line_errors(lineno, line))
 
 # For any "Bug: " line, next line is not blank
 if last_bug == last_lineno:

-- 
To view, visit https://gerrit.wikimedia.org/r/247047
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id27070a5db98959523fba15c65964c2384c8a79b
Gerrit-PatchSet: 1
Gerrit-Project: integration/jenkins
Gerrit-Branch: master
Gerrit-Owner: XZise

[MediaWiki-commits] [Gerrit] [FEAT] Rewrite commit message validator - change (integration/jenkins)

2015-10-17 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247056

Change subject: [FEAT] Rewrite commit message validator
..

[FEAT] Rewrite commit message validator

To properly store the status of the already interpreted commit message, like
already parsed change ids, a class can store that information without requiring
module wide variables.

This also converts all of the checks into yields so that it is almost nowhere
necessary to actually provide the line number it acts on. This changes a bit
the logic but also makes sure that the line numbers are now sequential.

It also does not compare the string to the previous situation but has
determined the paragraph borders before so that it can do paragraph specific
tests (e.g. complain about bug entries which are not in the last paragraph). A
paragraph in this case is a block of text separated by newlines.

Change-Id: I788ae0d12fd9ac5f0babf8fab8e0b3effe8eec68
---
M tests/test_commit-message-validator/check_message_errors.out
M tools/commit-message-validator.py
2 files changed, 102 insertions(+), 98 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/56/247056/1

diff --git a/tests/test_commit-message-validator/check_message_errors.out 
b/tests/test_commit-message-validator/check_message_errors.out
index a54cc06..f25764c 100644
--- a/tests/test_commit-message-validator/check_message_errors.out
+++ b/tests/test_commit-message-validator/check_message_errors.out
@@ -4,17 +4,20 @@
 Line 4: Use 'Bug: ' not 'Fixes:'
 Line 4: Expected one space after 'Bug:'
 Line 4: The bug ID is not a phabricator task: foo
+Line 4: Expected 'Bug:' only in last paragraph.
 Line 5: Use 'Bug: ' not 'Closes:'
 Line 5: Expected one space after 'Bug:'
 Line 5: The bug ID is not a phabricator task: bar
+Line 5: Expected 'Bug:' only in last paragraph.
 Line 6: Use 'Bug: ' not 'Task:'
+Line 6: Expected 'Bug:' only in last paragraph.
 Line 7: Expected 'Bug:' not 'BUG:'
 Line 7: Expected one space after 'Bug:'
 Line 7: The bug ID must be a phabricator task: 13
+Line 7: Expected 'Bug:' only in last paragraph.
 Line 8: The bug ID is not a phabricator task: T11 T12
-Line 8: Expected blank line before Bug:
-Line 9: Unexpected blank line after Bug:
-Line 12: Unexpected blank line after Bug:
-Line 12: Unexpected line between Bug: and Change-Id:
-Line 13: Extra Change-Id found, next at 14
-Line 15: Unexpected line after Change-Id
+Line 8: Expected 'Bug:' only in last paragraph.
+Line 10: Expected 'Bug:' only in last paragraph.
+Line 11: Expected 'Bug:' only in last paragraph.
+Line 14: Already found Change-Id at line 13
+Line 15: Unexpected line in last paragraph.
diff --git a/tools/commit-message-validator.py 
b/tools/commit-message-validator.py
index dacf03b..7b61296 100755
--- a/tools/commit-message-validator.py
+++ b/tools/commit-message-validator.py
@@ -13,7 +13,8 @@
 import subprocess
 
 
-def line_errors(lineno, line):
+class CheckMessage(object):
+
 """Check a commit message line to see if it has errors.
 
 Checks:
@@ -25,43 +26,103 @@
 - Exactly one task id on each Bug: line
 - No "Task: ", "Fixes: ", "Closes: " lines
 """
-# First line <=80
-if lineno == 0:
-if len(line) > 80:
-yield "First line should be <=80 characters"
-m = re.match(r'^T?\d+', line)
-if m:
-yield "Do not define bug in the header"
 
-# Second line blank
-elif lineno == 1:
-if line:
-yield "Second line should be empty"
+CHERRY_PICK_LINE = re.compile(r"^\(cherry picked from commit "
+  r"([a-fA-F0-9]{7,40})\)$")
 
-# No line >100
-elif len(line) > 100:
-yield "Line should be <=100 characters"
+def __init__(self, lines):
+self._lines = lines
+self._paragraphs = []
+last_start = 0
+for lineno, line in enumerate(lines):
+if not line:
+if last_start != lineno:
+self._paragraphs += [(last_start, lineno)]
+last_start = lineno + 1
+self._paragraphs += [(last_start, lineno)]
 
-m = re.match(r'^(bug|closes|fixes|task):(\W)*(.*)', line, re.IGNORECASE)
-if m:
+def check_line(self, lineno):
+line = self._lines[lineno]
+# First line <=80
 if lineno == 0:
-yield "Do not define bug in the header"
+if len(line) > 80:
+yield "First line should be <=80 characters"
+m = re.match(r'^T?(\d+)', line)
+if m:
+yield "Do not define bug in the header"
 
-if m.group(1).lower() == 'bug':
-if m.group(1) != 'Bug':
-yield "Expected 'Bug:' not '%s:'" % m.group(1)
+# Second line blank
+elif lineno == 1:
+if line:
+yield "Second line should be empty"
+
+# No line >100
+

[MediaWiki-commits] [Gerrit] [FEAT] Detect bug id duplicates - change (integration/jenkins)

2015-10-17 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247059

Change subject: [FEAT] Detect bug id duplicates
..

[FEAT] Detect bug id duplicates

It caches all bug ids either given at the beginning of the first line or via
`Bug:` (if the text is either a phabricator id or number) and warns when an id
was reused.

Change-Id: Ic20571654b0343b78b713fdfc3e23cdc5b2f75a4
---
M tools/commit-message-validator.py
1 file changed, 18 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/59/247059/1

diff --git a/tools/commit-message-validator.py 
b/tools/commit-message-validator.py
index 92e6738..9bf1b77 100755
--- a/tools/commit-message-validator.py
+++ b/tools/commit-message-validator.py
@@ -50,6 +50,7 @@
 m = re.match(r'^T?(\d+)', line)
 if m:
 yield "Do not define bug in the header"
+self._bugs[int(m.group(1))] = lineno
 
 # Second line blank
 elif lineno == 1:
@@ -82,8 +83,24 @@
 bug_id = m.group(3).strip()
 if bug_id.isdigit():
 yield "The bug ID must be a phabricator task: 
{0}".format(bug_id)
+# Phabricator IDs are 2000 higher than the Bugzilla IDs which
+# are referenced by "Bug: NUMBER".
+bug_id_int = int(bug_id) + 2000
 elif not bug_id.startswith('T') or not bug_id[1:].isdigit():
 yield "The bug ID is not a phabricator task: 
{0}".format(bug_id)
+bug_id_int = None
+else:
+bug_id_int = int(bug_id[1:])
+
+if bug_id_int is not None:
+if bug_id.startswith('0'):
+yield "The bug ID uses leading zeros: {0}".format(bug_id)
+
+if bug_id_int in self._bugs:
+yield "The bug ID already given in line {0}: T{1}".format(
+self._bugs[bug_id_int] + 1, bug_id_int)
+else:
+self._bugs[bug_id_int] = lineno
 else:
 is_bug = False
 
@@ -114,6 +131,7 @@
 
 def check(self):
 self._first_changeid = False
+self._bugs = {}
 for lineno in range(len(self._lines)):
 for e in self.check_line(lineno):
 yield 'Line {0}: {1}'.format(lineno + 1, e)

-- 
To view, visit https://gerrit.wikimedia.org/r/247059
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic20571654b0343b78b713fdfc3e23cdc5b2f75a4
Gerrit-PatchSet: 1
Gerrit-Project: integration/jenkins
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] config: Don't crash on later get_base_dir calls - change (pywikibot/core)

2015-10-16 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/246942

Change subject: [FIX] config: Don't crash on later get_base_dir calls
..

[FIX] config: Don't crash on later get_base_dir calls

All module variables starting with only one underscore get deleted after the
module's instantiation and with 2b07db2a the variable to store whether no user
config should be loaded was stored in such a variable and thus deleted so that
when `get_base_dir` was called afterwards it would crash because it is missing.

This patch renames the variable into using two underscores to still hide it's
presence but prevent it from deletion after the instantiation.

Conflicts:
pywikibot/config2.py

Change-Id: I538a73ff37da27a34ab732becafbf5a58b72a464
(cherry picked from commit bd44970f06c56626bb68c45a556bcb931f5480c6)
---
M pywikibot/config2.py
1 file changed, 8 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/42/246942/1

diff --git a/pywikibot/config2.py b/pywikibot/config2.py
index f9dcab8..f2d644f 100644
--- a/pywikibot/config2.py
+++ b/pywikibot/config2.py
@@ -60,9 +60,9 @@
 # names and some magic variables (like __name__)
 _imports = frozenset(name for name in globals() if not name.startswith('_'))
 
-_no_user_config = os.environ.get('PYWIKIBOT2_NO_USER_CONFIG')
-if _no_user_config == '0':
-_no_user_config = None
+__no_user_config = os.environ.get('PYWIKIBOT2_NO_USER_CONFIG')
+if __no_user_config == '0':
+__no_user_config = None
 
 
 class _ConfigurationDeprecationWarning(UserWarning):
@@ -297,8 +297,8 @@
 # check if user-config.py is in base_dir
 if not exists(base_dir):
 exc_text = "No user-config.py found in directory '%s'.\n" % base_dir
-if _no_user_config:
-if _no_user_config != '2':
+if __no_user_config:
+if __no_user_config != '2':
 print(exc_text)
 else:
 exc_text += "  Please check that user-config.py is stored in the 
correct location.\n"
@@ -905,8 +905,8 @@
 
 # Get the user files
 _thislevel = 0
-if _no_user_config:
-if _no_user_config != '2':
+if __no_user_config:
+if __no_user_config != '2':
 print("WARNING: Skipping loading of user-config.py.")
 _fns = []
 else:
@@ -1004,7 +1004,7 @@
 
 
 # Fix up default site
-if family == 'wikipedia' and mylang == 'language' and _no_user_config != '2':
+if family == 'wikipedia' and mylang == 'language' and __no_user_config != '2':
 print("WARNING: family and mylang are not set.\n"
   "Defaulting to family='test' and mylang='test'.")
 family = mylang = 'test'

-- 
To view, visit https://gerrit.wikimedia.org/r/246942
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I538a73ff37da27a34ab732becafbf5a58b72a464
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] config: Don't crash on later get_base_dir calls - change (pywikibot/core)

2015-10-16 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/246853

Change subject: [FIX] config: Don't crash on later get_base_dir calls
..

[FIX] config: Don't crash on later get_base_dir calls

All module variables starting with only one underscore get deleted after the
module's instantiation and with 2b07db2a the variable to store whether no user
config should be loaded was stored in such a variable and thus deleted so that
when `get_base_dir` was called afterwards it would crash because it is missing.

This patch renames the variable into using two underscores to still hide it's
presence but prevent it from deletion after the instantiation.

Change-Id: I538a73ff37da27a34ab732becafbf5a58b72a464
---
M pywikibot/config2.py
1 file changed, 8 insertions(+), 8 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/53/246853/1

diff --git a/pywikibot/config2.py b/pywikibot/config2.py
index 4cac315..8fd624e 100644
--- a/pywikibot/config2.py
+++ b/pywikibot/config2.py
@@ -70,9 +70,9 @@
 # names and some magic variables (like __name__)
 _imports = frozenset(name for name in globals() if not name.startswith('_'))
 
-_no_user_config = os.environ.get('PYWIKIBOT2_NO_USER_CONFIG')
-if _no_user_config == '0':
-_no_user_config = None
+__no_user_config = os.environ.get('PYWIKIBOT2_NO_USER_CONFIG')
+if __no_user_config == '0':
+__no_user_config = None
 
 
 class _ConfigurationDeprecationWarning(UserWarning):
@@ -323,8 +323,8 @@
 # check if user-config.py is in base_dir
 if not exists(base_dir):
 exc_text = "No user-config.py found in directory '%s'.\n" % base_dir
-if _no_user_config:
-if _no_user_config != '2':
+if __no_user_config:
+if __no_user_config != '2':
 output(exc_text)
 else:
 exc_text += "  Please check that user-config.py is stored in the 
correct location.\n"
@@ -935,8 +935,8 @@
 
 # Get the user files
 _thislevel = 0
-if _no_user_config:
-if _no_user_config != '2':
+if __no_user_config:
+if __no_user_config != '2':
 warning('Skipping loading of user-config.py.')
 _fns = []
 else:
@@ -1072,7 +1072,7 @@
 
 # Fix up default site
 if family == 'wikipedia' and mylang == 'language':
-if _no_user_config != '2':
+if __no_user_config != '2':
 warning('family and mylang are not set.\n'
 "Defaulting to family='test' and mylang='test'.")
 family = mylang = 'test'

-- 
To view, visit https://gerrit.wikimedia.org/r/246853
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I538a73ff37da27a34ab732becafbf5a58b72a464
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Test returned content first - change (integration/jenkins)

2015-10-16 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/247018

Change subject: [IMPROV] Test returned content first
..

[IMPROV] Test returned content first

Instead of just checking the exit code, it's more helpful to actually see the
output difference. For example when it excepts success it would only tell that
it got a failure but not what failed. If it expected a failure and would get
success it would show an empty string which implies that it erroneously
succeeded.

Change-Id: If384513b9f952b91f44dfd0b5a3144abb46f3be7
---
M tests/test_commit-message-validator.py
1 file changed, 2 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/18/247018/1

diff --git a/tests/test_commit-message-validator.py 
b/tests/test_commit-message-validator.py
index b593fa2..6417cba 100644
--- a/tests/test_commit-message-validator.py
+++ b/tests/test_commit-message-validator.py
@@ -34,10 +34,9 @@
 try:
 out = StringIO()
 sys.stdout = out
-self.assertEqual(
-1 if expected else 0,
-cmv.check_message(msg.splitlines()))
+exit_code = cmv.check_message(msg.splitlines())
 self.assertEqual(expected, out.getvalue())
+self.assertEqual(exit_code, 1 if expected else 0)
 finally:
 sys.stdout = saved_stdout
 return test

-- 
To view, visit https://gerrit.wikimedia.org/r/247018
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If384513b9f952b91f44dfd0b5a3144abb46f3be7
Gerrit-PatchSet: 1
Gerrit-Project: integration/jenkins
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] cosmetic_changes: Detect useless space at line end - change (pywikibot/core)

2015-10-15 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/246795

Change subject: [FIX] cosmetic_changes: Detect useless space at line end
..

[FIX] cosmetic_changes: Detect useless space at line end

It only trimmed useless spaces at the string end but not at each line end. And
instead of using two regexes to search for them it combines both into one regex
to only need to search once. It also adds the site parameter as the call has
been changed.

Change-Id: I8cf45d76441af49756f5933a651d07ecf61f2f82
---
M pywikibot/cosmetic_changes.py
M tests/cosmetic_changes_tests.py
2 files changed, 6 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/95/246795/1

diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py
index 93a19ba..3010ef3 100755
--- a/pywikibot/cosmetic_changes.py
+++ b/pywikibot/cosmetic_changes.py
@@ -617,13 +617,11 @@
 
 def removeUselessSpaces(self, text):
 """Cleanup multiple or trailing spaces."""
-multipleSpacesR = re.compile('  +')
-spaceAtLineEndR = re.compile(' $')
 exceptions = ['comment', 'math', 'nowiki', 'pre', 'startspace', 
'table']
 if self.site.sitename != 'wikipedia:cs':
 exceptions.append('template')
-text = textlib.replaceExcept(text, multipleSpacesR, ' ', exceptions)
-text = textlib.replaceExcept(text, spaceAtLineEndR, '', exceptions)
+text = textlib.replaceExcept(text, r'(?m) +( |$)', r'\1', exceptions,
+ site=self.site)
 return text
 
 def removeNonBreakingSpaceBeforePercent(self, text):
diff --git a/tests/cosmetic_changes_tests.py b/tests/cosmetic_changes_tests.py
index c4fdc3e..17e270d 100644
--- a/tests/cosmetic_changes_tests.py
+++ b/tests/cosmetic_changes_tests.py
@@ -52,6 +52,10 @@
 """Test removeUselessSpaces method."""
 self.assertEqual('Foo bar',
  self.cct.removeUselessSpaces('Foo  bar '))
+self.assertEqual('Foo bar',
+ self.cct.removeUselessSpaces('Foo  bar   '))
+self.assertEqual('Foo bar\nsna fu',
+ self.cct.removeUselessSpaces('Foo  bar \nsna  fu  '))
 # inside comments
 self.assertEqual('',
  self.cct.removeUselessSpaces(''))

-- 
To view, visit https://gerrit.wikimedia.org/r/246795
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8cf45d76441af49756f5933a651d07ecf61f2f82
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] cosmetic_changes: Replace helper method - change (pywikibot/core)

2015-10-15 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/246772

Change subject: [FEAT] cosmetic_changes: Replace helper method
..

[FEAT] cosmetic_changes: Replace helper method

Instead of using `replaceExcept` everywhere and to handle `text` manually this
adds a `replace` method which edits an attribute and thus it's not necessary
for the caller to provide the text or store its result. It also uses a sensible
set of default exceptions which can be extended or overwritten in case they
don't actually match the need.

Change-Id: I9f570f4d424ab6a5d6fdcb17268cdf4e164838ea
---
M pywikibot/cosmetic_changes.py
M tests/cosmetic_changes_tests.py
2 files changed, 129 insertions(+), 139 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/72/246772/1

diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py
index 6b4af85..f546de6 100755
--- a/pywikibot/cosmetic_changes.py
+++ b/pywikibot/cosmetic_changes.py
@@ -71,8 +71,9 @@
 import pywikibot
 
 from pywikibot import config, textlib
-from pywikibot.tools import deprecate_arg, first_lower, first_upper
-from pywikibot.tools import MediaWikiVersion
+from pywikibot.tools import (
+add_full_name, deprecate_arg, first_lower, first_upper, MediaWikiVersion,
+)
 
 
 # This is from interwiki.py;
@@ -198,6 +199,9 @@
 
 """Cosmetic changes toolkit."""
 
+DEFAULT_EXCEPTIONS = frozenset(['nowiki', 'comment', 'math', 'pre',
+'source', 'startspace'])
+
 @deprecate_arg('debug', 'diff')
 def __init__(self, site, diff=False, redirect=False, namespace=None,
  pageTitle=None, ignore=CANCEL_ALL):
@@ -279,7 +283,43 @@
 pywikibot.showDiff(text, new_text)
 return new_text
 
-def fixSelfInterwiki(self, text):
+def replace(self, pattern, replacement, exceptions=None,
+case_insensitive=False, **kwargs):
+"""
+Replace the pattern with the replacement on the own text.
+
+It is by default using C{DEFAULT_EXCEPTIONS} and it is possible to
+overwrite that set by using a frozenset or tuple. If it's anything else
+it'll also use these exceptions togehter with the default ones.
+"""
+if exceptions is None:
+exceptions = set()
+if not isinstance(exceptions, (frozenset, tuple)):
+exceptions = self.DEFAULT_EXCEPTIONS | set(exceptions)
+
+self._text = textlib.replaceExcept(
+self._text, pattern, replacement, exceptions, case_insensitive,
+site=self.site, **kwargs)
+
+@add_full_name
+def optional_text(func):
+"""Wrapper to cache the text in it's own object for replace."""
+def wrapped(self, text):
+if text is not None and hasattr(self, '_text'):
+old_text = self._text
+else:
+old_text = None
+self._text = text
+result = func(self)
+assert result is None
+text = self._text
+if old_text is not None:
+self._text = old_text
+return text
+return wrapped
+
+@optional_text
+def fixSelfInterwiki(self):
 """
 Interwiki links to the site itself are displayed like local links.
 
@@ -288,8 +328,7 @@
 if not self.talkpage and pywikibot.calledModuleName() != 'interwiki':
 interwikiR = re.compile(r'\[\[%s\s?:([^\[\]\n]*)\]\]'
 % self.site.code)
-text = interwikiR.sub(r'[[\1]]', text)
-return text
+self._text = interwikiR.sub(r'[[\1]]', self._text)
 
 def standardizePageFooter(self, text):
 """
@@ -402,15 +441,14 @@
 template_subpage=subpage)
 return text
 
-def translateAndCapitalizeNamespaces(self, text):
+@optional_text
+def translateAndCapitalizeNamespaces(self):
 """Use localized namespace names."""
 # arz uses english stylish codes
 if self.site.sitename == 'wikipedia:arz':
-return text
+return
 family = self.site.family
 # wiki links aren't parsed here.
-exceptions = ['nowiki', 'comment', 'math', 'pre']
-
 for namespace in self.site.namespaces.values():
 if namespace.id in (0, 2, 3):
 # skip main (article) namespace
@@ -439,32 +477,27 @@
 namespaces[i] = item
 namespaces.append(first_lower(thisNs))
 if thisNs and namespaces:
-text = textlib.replaceExcept(
-text,
+self.replace(
 r'\[\[\s*(%s) *:(?P.*?)\]\]'
 % '|'.join(namespaces),
-r'[[%s:\g]]' % thisNs,
-exceptions)
-return text
+

[MediaWiki-commits] [Gerrit] [FEAT] i18n: Support specific plural messages - change (pywikibot/core)

2015-10-13 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/245886

Change subject: [FEAT] i18n: Support specific plural messages
..

[FEAT] i18n: Support specific plural messages

This adds support for translations which use specific plural texts for specific
values.

Bug: T115297
Change-Id: I5372ed4ef3343cbbab1b33efcee0cf707e50ae24
---
M pywikibot/i18n.py
M tests/i18n_tests.py
2 files changed, 44 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/86/245886/1

diff --git a/pywikibot/i18n.py b/pywikibot/i18n.py
index 3bd9e9e..944958b 100644
--- a/pywikibot/i18n.py
+++ b/pywikibot/i18n.py
@@ -356,11 +356,24 @@
 'an int', 1)
 num = int(num)
 
+entries = [None] + re.split(r'\|(?: *(\d+) *= *)?', variants)
+plural_entries = []
+specific_entries = {}
+for index in range(0, len(entries), 2):
+if entries[index] is not None:
+specific_entries[int(entries[index])] = entries[index + 1]
+else:
+assert not specific_entries, \
+'generic entries defined after specific in 
"{0}"'.format(variants)
+plural_entries += [entries[index + 1]]
+
+if num in specific_entries:
+return specific_entries[num]
+
 index = plural_value(num)
 if rule['nplurals'] == 1:
 assert index == 0
 
-plural_entries = variants.split('|')
 if index >= len(plural_entries):
 raise IndexError(
 'requested plural {0} for {1} but only {2} ("{3}") '
diff --git a/tests/i18n_tests.py b/tests/i18n_tests.py
index 0ffb661..9b68faf 100644
--- a/tests/i18n_tests.py
+++ b/tests/i18n_tests.py
@@ -413,6 +413,36 @@
 self.assertIn('dummy output: ', self.output_text)
 
 
+class TestExtractPlural(TestCase):
+
+"""Test extracting plurals from a dummy string."""
+
+net = False
+
+def test_standard(self):
+"""Test default usage using a dict and no specific plurals."""
+self.assertEqual(
+i18n._extract_plural('en', '{{PLURAL:foo|one|other}}', {'foo': 
42}),
+'other')
+self.assertEqual(
+i18n._extract_plural('en', '{{PLURAL:foo|one|other}}', {'foo': 1}),
+'one')
+self.assertEqual(
+i18n._extract_plural('en', '{{PLURAL:foo|one|other}}', {'foo': 0}),
+'other')
+
+def test_specific(self):
+"""Test using a specific plural."""
+self.assertEqual(
+i18n._extract_plural('en', '{{PLURAL:foo|one|other|12=dozen}}',
+ {'foo': 42}),
+'other')
+self.assertEqual(
+i18n._extract_plural('en', '{{PLURAL:foo|one|other|12=dozen}}',
+ {'foo': 12}),
+'dozen')
+
+
 if __name__ == '__main__':
 try:
 unittest.main()

-- 
To view, visit https://gerrit.wikimedia.org/r/245886
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5372ed4ef3343cbbab1b33efcee0cf707e50ae24
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] i18n: Deprecate list as code in twtranslate - change (pywikibot/core)

2015-10-12 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/245610

Change subject: [IMPROV] i18n: Deprecate list as code in twtranslate
..

[IMPROV] i18n: Deprecate list as code in twtranslate

`twtranslate` supports a list as code in order to change it and return the
actually used code. This was only used for `twntranslate` and shouldn't be used
anymore.

Change-Id: I13d45e39ea52d77983fb4e654f8c0772f0263306
---
M pywikibot/i18n.py
1 file changed, 8 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/10/245610/1

diff --git a/pywikibot/i18n.py b/pywikibot/i18n.py
index 58e23fd..3bd9e9e 100644
--- a/pywikibot/i18n.py
+++ b/pywikibot/i18n.py
@@ -33,6 +33,7 @@
 import pkgutil
 
 from collections import defaultdict, Mapping
+from warnings import warn
 
 import pywikibot
 
@@ -554,7 +555,9 @@
 ... % {'descr': 'seulement'})
 'Robot: Changer seulement quelques pages.'
 
-@param code: The language code
+@param code: When it's a site it's using the code attribute and otherwise 
it
+is using the value directly.
+@type code: BaseSite or str
 @param twtitle: The TranslateWiki string title, in - format
 @param parameters: For passing parameters. It should be a mapping but for
 backwards compatibility can also be a list, tuple or a single value.
@@ -583,6 +586,10 @@
 lang = code.code
 # check whether we need the language code back
 elif isinstance(code, list):
+# For backwards compatibility still support lists, when twntranslate
+# was not deprecated and needed a way to get the used language code 
back
+warn('The code argument should not be a list but either a BaseSite or '
+ 'a str/unicode.', DeprecationWarning, 2)
 lang = code.pop()
 code_needed = True
 else:

-- 
To view, visit https://gerrit.wikimedia.org/r/245610
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I13d45e39ea52d77983fb4e654f8c0772f0263306
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Provide helper for generated tests - change (pywikibot/core)

2015-10-11 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244966

Change subject: [IMPROV] Provide helper for generated tests
..

[IMPROV] Provide helper for generated tests

With 4ec75cd7 all tests which were generated using multiple sites are using
`__name__` now. But there are other cases which dynamically create tests and
may not add that property. So `MetaTestCaseClass` now supports `add_test` which
adds the value to the `dct` and sets `__name__` and allows to appends something
to a docstring (or to overwrite the docstring).

Change-Id: I7797ee461d82d764c32e92f773e9c7433fda24bd
---
M tests/aspects.py
M tests/date_tests.py
M tests/l10n_tests.py
M tests/logentry_tests.py
M tests/script_tests.py
M tests/tools_tests.py
6 files changed, 34 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/66/244966/1

diff --git a/tests/aspects.py b/tests/aspects.py
index 8e94e38..2de6fbe 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -848,9 +848,8 @@
 # create test methods processed by unittest
 for (key, sitedata) in dct['sites'].items():
 test_name = test + '_' + key.replace('-', '_')
-
-dct[test_name] = wrap_method(key, sitedata, dct[test])
-dct[test_name].__name__ = str(test_name)
+cls.add_test(dct, test_name,
+ wrap_method(key, sitedata, dct[test]))
 
 if key in dct.get('expected_failures', []):
 dct[test_name] = unittest.expectedFailure(dct[test_name])
@@ -866,6 +865,21 @@
 bases = (subclass, ) + bases
 return bases
 
+@staticmethod
+def add_test(dct, test_name, method, doc=None, doc_suffix=None):
+"""Add a method to a dictionary and set its name and documention."""
+dct[test_name] = method
+# it's explicitly using str() because __name__ must be str
+dct[test_name].__name__ = str(test_name)
+if doc_suffix:
+if not doc:
+doc = method.__doc__
+assert doc[-1] == '.'
+doc = doc[:-1] + ' ' + doc_suffix + '.'
+
+if doc:
+dct[test_name].__doc__ = doc
+
 
 @add_metaclass
 class TestCase(TestTimerMixin, TestCaseBase):
diff --git a/tests/date_tests.py b/tests/date_tests.py
index c8fe5ab..45179ad 100644
--- a/tests/date_tests.py
+++ b/tests/date_tests.py
@@ -49,10 +49,8 @@
 return testMapEntry
 
 for formatname in date.formats:
-# it's explicitly using str() because __name__ must be str
-test_name = str('test_' + formatname)
-dct[test_name] = test_method(formatname)
-dct[test_name].__name__ = test_name
+cls.add_test(dct, 'test_' + formatname, test_method(formatname))
+
 return type.__new__(cls, name, bases, dct)
 
 
diff --git a/tests/l10n_tests.py b/tests/l10n_tests.py
index d662647..9031dc6 100644
--- a/tests/l10n_tests.py
+++ b/tests/l10n_tests.py
@@ -68,9 +68,11 @@
 for code in codes:
 current_site = pywikibot.Site(code, dct['family'])
 test_name = ("test_%s_%s" % (package, code)).replace('-', '_')
-dct[test_name] = test_method(current_site)
-dct[test_name].__doc__ = 'Test {0} with language {1}'.format(
-package, code)
+cls.add_test(
+dct, test_name, test_method(current_site),
+doc_suffix='{0} and language {1}'.format(
+package, code))
+
 return super(TestValidTemplateMeta, cls).__new__(cls, name, bases, dct)
 
 
diff --git a/tests/logentry_tests.py b/tests/logentry_tests.py
index b43c770..df471b7 100644
--- a/tests/logentry_tests.py
+++ b/tests/logentry_tests.py
@@ -107,8 +107,8 @@
 
 # create test methods for the support logtype classes
 for logtype in LogEntryFactory._logtypes:
-test_name = str('test_%sEntry' % logtype.title())
-dct[test_name] = test_method(logtype)
+cls.add_test(dct, 'test_%sEntry' % logtype.title(),
+ test_method(logtype))
 
 return super(TestLogentriesMeta, cls).__new__(cls, name, bases, dct)
 
diff --git a/tests/script_tests.py b/tests/script_tests.py
index 66bd657..c62011d 100644
--- a/tests/script_tests.py
+++ b/tests/script_tests.py
@@ -332,7 +332,7 @@
 return test_skip_script
 return testScript
 
-argument = dct['_argument']
+argument = '-' + dct['_argument']
 
 for script_name in script_list:
 # force login to be the first, alphabetically, so the login
@@ -345,18 +345,14 @@
 else:
 test_name = 'test_' + script_name
 
-# it's explicitly using str() because __name__ must be str
-test_name =

[MediaWiki-commits] [Gerrit] [IMPROV] i18n_tests: Verify plural rules - change (pywikibot/core)

2015-10-11 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244967

Change subject: [IMPROV] i18n_tests: Verify plural rules
..

[IMPROV] i18n_tests: Verify plural rules

This verifies the consistency of all plural rules. Static plural rules must
have `nplurals` set to 1 and return 0 for `plural`. If the plural is a callable
it'll test all values from 0 until `max_num` and check that the indexes
returned are greater than 0 but also below the number of plurals. That maximum
number thus must then cover all possible plurals for all languages.

Change-Id: If2a665fa047630de30ac51a108144affb08afa16
---
M tests/i18n_tests.py
1 file changed, 53 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/67/244967/1

diff --git a/tests/i18n_tests.py b/tests/i18n_tests.py
index 0ffb661..4330e4d 100644
--- a/tests/i18n_tests.py
+++ b/tests/i18n_tests.py
@@ -16,8 +16,9 @@
 
 from tests.aspects import (
 unittest, TestCase, DefaultSiteTestCase, PwbTestCase,
-AutoDeprecationTestCase,
+AutoDeprecationTestCase, MetaTestCaseClass,
 )
+from tests.utils import add_metaclass
 
 
 class TestTranslate(TestCase):
@@ -413,6 +414,57 @@
 self.assertIn('dummy output: ', self.output_text)
 
 
+class MetaPluralRulesTest(MetaTestCaseClass):
+
+"""Metaclass to test each plural rule in separate tests."""
+
+def __new__(cls, name, bases, dct):
+"""Create a new test case which tests all plural rules."""
+def create_test(rule):
+def test_static_rule(self):
+"""Test a rule which is just one integer."""
+self.assertEqual(rule['nplurals'], 1)
+self.assertEqual(rule['plural'], 0)
+
+def test_callable_rule(self):
+"""Test a rule which is callable."""
+# in theory a static rule could be also callable
+self.assertGreater(rule['nplurals'], 0)
+num_plurals = set()
+for num in range(self.max_num + 1):
+index = rule['plural'](num)
+self.assertLess(index, rule['nplurals'],
+msg='Plural for {0} created an index {1} '
+'(greater than {2})'.format(num, index,
+
rule['nplurals']))
+num_plurals.add(index)
+self.assertCountEqual(num_plurals, 
list(range(rule['nplurals'])))
+
+# Don't already fail on creation
+if callable(rule.get('plural')):
+return test_callable_rule
+else:
+return test_static_rule
+
+for lang, rule in plural.plural_rules.items():
+cls.add_test(dct, 'test_{0}'.format(lang), create_test(rule),
+ doc_suffix='for "{0}"'.format(lang))
+return super(MetaPluralRulesTest, cls).__new__(cls, name, bases, dct)
+
+
+@add_metaclass
+class TestPluralRules(TestCase):
+
+"""Test the consistency of the plural rules."""
+
+__metaclass__ = MetaPluralRulesTest
+
+net = False
+# for callable plural rules it'll test up until this number, this number
+# must cause to create all plurals in all dynamic languages
+max_num = 1000
+
+
 if __name__ == '__main__':
 try:
 unittest.main()

-- 
To view, visit https://gerrit.wikimedia.org/r/244967
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If2a665fa047630de30ac51a108144affb08afa16
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] proofreadpage_tests: Preserve page order - change (pywikibot/core)

2015-10-11 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/245141

Change subject: [FIX] proofreadpage_tests: Preserve page order
..

[FIX] proofreadpage_tests: Preserve page order

The test for redlinks in an index page have been added in f77fb56f but they
were using `dict.values` which is not stable and thus might change the order.

Change-Id: I4e9d9ac72ad034ed232f8c0338423c33a0123e85
---
M tests/proofreadpage_tests.py
1 file changed, 7 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/41/245141/1

diff --git a/tests/proofreadpage_tests.py b/tests/proofreadpage_tests.py
index 67927fd..25eaf4c 100644
--- a/tests/proofreadpage_tests.py
+++ b/tests/proofreadpage_tests.py
@@ -542,15 +542,16 @@
 cached = True
 
 with_redlink = {
-'title': {'blue': 'Page:Pywikibot test page 1/1',
-  'red': 'Page:Pywikibot test page 2/2',
-  },
+'title': ['Page:Pywikibot test page 1/1',
+  'Page:Pywikibot test page 2/2',
+  ],
+'missing': 'Page:Pywikibot test page 2/2',
 'index': 'Index:Pywikibot test page 1'
 }
 
 def test_index_redlink(self):
 """Test index property with redlink."""
-page = ProofreadPage(self.site, self.with_redlink['title']['red'])
+page = ProofreadPage(self.site, self.with_redlink['missing'])
 index_page = IndexPage(self.site, self.with_redlink['index'])
 self.assertEqual(page.index, index_page)
 
@@ -558,7 +559,7 @@
 """Test IndexPage page get_page_number functions with redlinks."""
 index_page = IndexPage(self.site, self.with_redlink['index'])
 
-for title in self.with_redlink['title'].values():
+for title in self.with_redlink['title']:
 p = ProofreadPage(self.site, title)
 n = index_page.get_number(p)
 self.assertEqual(index_page.get_page(n), p)
@@ -567,7 +568,7 @@
 """Test Index page generator with redlinks."""
 index_page = IndexPage(self.site, self.with_redlink['index'])
 proofread_pages = [ProofreadPage(self.site, page_title) for
-   page_title in self.with_redlink['title'].values()]
+   page_title in self.with_redlink['title']]
 
 # Check start/end limits.
 self.assertRaises(ValueError, index_page.page_gen, -1, 2)

-- 
To view, visit https://gerrit.wikimedia.org/r/245141
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4e9d9ac72ad034ed232f8c0338423c33a0123e85
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] proofreadpage_tests: Prepare redlink tests - change (pywikibot/core)

2015-10-11 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/245142

Change subject: [FIX] proofreadpage_tests: Prepare redlink tests
..

[FIX] proofreadpage_tests: Prepare redlink tests

The test for redlinks in an index page have been added in f77fb56f but they
were using `dict.values` which is not stable and thus might change the order.
Additionally this is preparing the test class by creating the page instances
once.

Change-Id: I55084049460f39a27e36ff9b0f6a3e544db68d04
---
M tests/proofreadpage_tests.py
1 file changed, 22 insertions(+), 24 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/42/245142/1

diff --git a/tests/proofreadpage_tests.py b/tests/proofreadpage_tests.py
index 67927fd..b1c5b4c 100644
--- a/tests/proofreadpage_tests.py
+++ b/tests/proofreadpage_tests.py
@@ -541,41 +541,39 @@
 
 cached = True
 
-with_redlink = {
-'title': {'blue': 'Page:Pywikibot test page 1/1',
-  'red': 'Page:Pywikibot test page 2/2',
-  },
-'index': 'Index:Pywikibot test page 1'
-}
+index_name = 'Index:Pywikibot test page 1'
+page_names = ['Page:Pywikibot test page 1/1',
+  'Page:Pywikibot test page 2/2',
+  ]
+missing_name = 'Page:Pywikibot test page 2/2'
+
+@classmethod
+def setUpClass(cls):
+"""Prepare tests by creating page instances."""
+super(TestIndexPageMappingsRedlinks, cls).setUpClass()
+cls.index = IndexPage(cls.site, cls.index_name)
+cls.pages = [ProofreadPage(cls.site, page) for page in cls.page_names]
+cls.missing = ProofreadPage(cls.site, cls.missing_name)
 
 def test_index_redlink(self):
 """Test index property with redlink."""
-page = ProofreadPage(self.site, self.with_redlink['title']['red'])
-index_page = IndexPage(self.site, self.with_redlink['index'])
-self.assertEqual(page.index, index_page)
+self.assertEqual(self.missing.index, self.index)
 
 def test_get_page_and_number_redlink(self):
 """Test IndexPage page get_page_number functions with redlinks."""
-index_page = IndexPage(self.site, self.with_redlink['index'])
-
-for title in self.with_redlink['title'].values():
-p = ProofreadPage(self.site, title)
-n = index_page.get_number(p)
-self.assertEqual(index_page.get_page(n), p)
+for page in self.pages:
+n = self.index.get_number(page)
+self.assertEqual(self.index.get_page(n), page)
 
 def test_page_gen_redlink(self):
 """Test Index page generator with redlinks."""
-index_page = IndexPage(self.site, self.with_redlink['index'])
-proofread_pages = [ProofreadPage(self.site, page_title) for
-   page_title in self.with_redlink['title'].values()]
-
 # Check start/end limits.
-self.assertRaises(ValueError, index_page.page_gen, -1, 2)
-self.assertRaises(ValueError, index_page.page_gen, 1, -1)
-self.assertRaises(ValueError, index_page.page_gen, 2, 1)
+self.assertRaises(ValueError, self.index.page_gen, -1, 2)
+self.assertRaises(ValueError, self.index.page_gen, 1, -1)
+self.assertRaises(ValueError, self.index.page_gen, 2, 1)
 
-gen = index_page.page_gen(1, None, filter_ql=range(5))
-self.assertEqual(list(gen), proofread_pages)
+gen = self.index.page_gen(1, None, filter_ql=range(5))
+self.assertEqual(list(gen), self.pages)
 
 
 if __name__ == '__main__':

-- 
To view, visit https://gerrit.wikimedia.org/r/245142
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I55084049460f39a27e36ff9b0f6a3e544db68d04
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] generate_user_files: Duplicated ERROR prefix - change (pywikibot/core)

2015-10-11 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/245197

Change subject: [FIX] generate_user_files: Duplicated ERROR prefix
..

[FIX] generate_user_files: Duplicated ERROR prefix

In a3765211 the prints were in the script were changed to `pywikibot.error` and
similar. One call was overlooked and continued to use the ERROR prefix in the
message. It also adds the exception message to that message so that it's
easier to figure out what went wrong.

Change-Id: I8ebc59d3cc65b0bec0e453e0a63b5d4bcef7bfff
---
M generate_user_files.py
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/97/245197/1

diff --git a/generate_user_files.py b/generate_user_files.py
index a66bc2d..4455862 100755
--- a/generate_user_files.py
+++ b/generate_user_files.py
@@ -59,8 +59,8 @@
 else:
 try:
 os.mkdir(new_base, pywikibot.config2.private_files_permission)
-except Exception:
-pywikibot.error("ERROR: directory creation failed")
+except Exception as e:
+pywikibot.error('directory creation failed: {0}'.format(e))
 continue
 pywikibot.output("Created new directory.")
 break

-- 
To view, visit https://gerrit.wikimedia.org/r/245197
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8ebc59d3cc65b0bec0e453e0a63b5d4bcef7bfff
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] cosmetic_changes: Generate header replacement with ... - change (pywikibot/core)

2015-10-10 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244877

Change subject: [IMPROV] cosmetic_changes: Generate header replacement with 
format
..

[IMPROV] cosmetic_changes: Generate header replacement with format

Instead of using the %-notation which doesn't allow reusing indexed parameters
it can use `str.format` to generate the header wiki text.

Change-Id: I04a627e2d3f99ca093c1970a7ffa7d9b4f1630a1
---
M pywikibot/cosmetic_changes.py
1 file changed, 1 insertion(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/77/244877/1

diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py
index 485769d..44ffca7 100755
--- a/pywikibot/cosmetic_changes.py
+++ b/pywikibot/cosmetic_changes.py
@@ -749,12 +749,11 @@
  exceptions)
 # a header where only spaces are in the same line
 for level in range(1, 7):
-equals = '\\1%s \\2 %s\\3' % ("=" * level, "=" * level)
 text = textlib.replaceExcept(
 text,
 r'(?i)([\r\n]) * *([^<]+?) * *([\r\n])'
 % (level, level),
-r'%s' % equals,
+r'\1{0} \2 {0}\3'.format('=' * level),
 exceptions)
 # TODO: maybe we can make the bot replace  tags with \r\n's.
 return text

-- 
To view, visit https://gerrit.wikimedia.org/r/244877
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I04a627e2d3f99ca093c1970a7ffa7d9b4f1630a1
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] cosmetic_changes: Implement external link to wikilink - change (pywikibot/core)

2015-10-10 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244880

Change subject: [FIX] cosmetic_changes: Implement external link to wikilink
..

[FIX] cosmetic_changes: Implement external link to wikilink

The feature to replace external links into a wikilink when it pointed to the
same wiki was introduced in compat as of f2645f85. But it had been disabled
since its addition. Later it was, still disabled, ported to core in 6ac688fe.

This now enables an improved version which does not allow the delimiter in the
text itself so that links to diffs for example won't be changed. Also instead
of using the assumption that the URL is `code.family` it's iterating over all
the URLs which are also available for `Family.from_url` and `Site(url=…)`.

Change-Id: Id06b256e2005f5730fbfbaa96cd6690ec4de1789
---
M pywikibot/cosmetic_changes.py
M pywikibot/family.py
M tests/cosmetic_changes_tests.py
3 files changed, 59 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/80/244880/1

diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py
index 485769d..939963a 100755
--- a/pywikibot/cosmetic_changes.py
+++ b/pywikibot/cosmetic_changes.py
@@ -696,16 +696,31 @@
 
 # from fixes.py
 def fixSyntaxSave(self, text):
+def replace_link(match):
+replacement = '[[' + match.group('link')
+if match.group('title'):
+replacement += '|' + match.group('title')
+return replacement + ']]'
+
 exceptions = ['nowiki', 'comment', 'math', 'pre', 'source',
   'startspace']
 # link to the wiki working on
-# TODO: disable this for difflinks and titled links,
-# to prevent edits like this:
-# 
https://de.wikipedia.org/w/index.php?title=Wikipedia%3aVandalismusmeldung=103109563=103109271
-#text = textlib.replaceExcept(text,
-# 
r'\[https?://%s\.%s\.org/wiki/(?P\S+)\s+(?P.+?)\s?\]'
-# % (self.site.code, 
self.site.family.name),
-# r'[[\g|\g]]', exceptions)
+for suffix in self.site._interwiki_urls():
+for protocol in ('http', 'https'):
+url = self.site.base_url(suffix, protocol)
+# Only include links which don't include the separator as
+# the wikilink won't support additional parameters
+if '?' in url:
+separator = '&'
+else:
+separator = '?'
+# Match first a non space in the title to prevent that multiple
+# spaces at the end without title will be matched by it
+text = textlib.replaceExcept(
+text,
+r'\[\[?' + re.escape(url) + r'(?P[^' + separator +
+r']+?)(\s+(?P[^\s].*?))?\s*\]\]?',
+replace_link, exceptions, site=self.site)
 # external link in/starting with double brackets
 text = textlib.replaceExcept(
 text,
diff --git a/pywikibot/family.py b/pywikibot/family.py
index b4c5850..862f766 100644
--- a/pywikibot/family.py
+++ b/pywikibot/family.py
@@ -1059,17 +1059,28 @@
 # Override this ONLY if the wiki family requires a path prefix
 return ''
 
-def _hostname(self, code):
+def _hostname(self, code, protocol=None):
 """Return the protocol and hostname."""
-protocol = self.protocol(code)
+if protocol is None:
+protocol = self.protocol(code)
 if protocol == 'https':
 host = self.ssl_hostname(code)
 else:
 host = self.hostname(code)
 return protocol, host
 
-def base_url(self, code, uri):
-protocol, host = self._hostname(code)
+def base_url(self, code, uri, protocol=None):
+"""
+Create the full URL.
+
+@param code: The site code
+@param uri: The absolute path after the hostname
+@param protocol: The protocol which is used. If None it'll determine 
the
+protocol from the code.
+@return: The full URL
+@rtype: str
+"""
+protocol, host = self._hostname(code, protocol)
 if protocol == 'https':
 uri = self.ssl_pathprefix(code) + uri
 return urlparse.urljoin('{0}://{1}'.format(protocol, host), uri)
diff --git a/tests/cosmetic_changes_tests.py b/tests/cosmetic_changes_tests.py
index a591f18..a62d592 100644
--- a/tests/cosmetic_changes_tests.py
+++ b/tests/cosmetic_changes_tests.py
@@ -91,6 +91,28 @@
 
 def test_fixSyntaxSave(self):
 """Test fixSyntaxSave method."""
+# necessary as the fixer needs the article path to fix it
+self.cct.site._siteinfo._cache['general'] = (
+{'articlepath': '/wiki/$1'}, True)

[MediaWiki-commits] [Gerrit] [IMPROV] Split cosmetic changes tests into dry and live - change (pywikibot/core)

2015-10-09 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244659

Change subject: [IMPROV] Split cosmetic changes tests into dry and live
..

[IMPROV] Split cosmetic changes tests into dry and live

Most of the tests work fine with just the basic `DrySite` and can be run on a
dry wiki.

Change-Id: I981c651537647f66eab22cd77d27c1a7b163c88f
---
M tests/cosmetic_changes_tests.py
1 file changed, 106 insertions(+), 94 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/59/244659/1

diff --git a/tests/cosmetic_changes_tests.py b/tests/cosmetic_changes_tests.py
index 209d27b..a591f18 100644
--- a/tests/cosmetic_changes_tests.py
+++ b/tests/cosmetic_changes_tests.py
@@ -28,12 +28,118 @@
 cls.cct = CosmeticChangesToolkit(cls.site, namespace=0,
  pageTitle='Test')
 
+
+class TestDryCosmeticChanges(TestCosmeticChanges):
+
+"""Test cosmetic_changes not requiring a live wiki."""
+
+dry = True
+
 def test_fixSelfInterwiki(self):
 """Test fixSelfInterwiki method."""
 self.assertEqual('[[Foo bar]]',
  self.cct.fixSelfInterwiki('[[de:Foo bar]]'))
 self.assertEqual('[[en:Foo bar]]',
  self.cct.fixSelfInterwiki('[[en:Foo bar]]'))
+
+def test_resolveHtmlEntities(self):
+"""Test resolveHtmlEntities method."""
+self.assertEqual(
+'## #0##x',
+
self.cct.resolveHtmlEntities('#'))
+
+def test_removeUselessSpaces(self):
+"""Test removeUselessSpaces method."""
+self.assertEqual('Foo bar',
+ self.cct.removeUselessSpaces('Foo  bar '))
+# inside comments
+self.assertEqual('',
+ self.cct.removeUselessSpaces(''))
+# startspace
+self.assertEqual(' Foo  bar ',
+ self.cct.removeUselessSpaces(' Foo  bar '))
+
+def test_removeNonBreakingSpaceBeforePercent(self):
+"""Test removeNonBreakingSpaceBeforePercent method."""
+self.assertEqual(
+'42 %', self.cct.removeNonBreakingSpaceBeforePercent('42%'))
+
+def test_cleanUpSectionHeaders(self):
+"""Test cleanUpSectionHeaders method."""
+self.assertEqual('=== Header ===\n',
+ self.cct.cleanUpSectionHeaders('===Header===\n'))
+
+def test_putSpacesInLists(self):
+"""Test putSpacesInLists method."""
+self.assertEqual('* Foo bar',
+ self.cct.putSpacesInLists('*Foo bar'))
+self.assertEqual('** Foo bar',
+ self.cct.putSpacesInLists('**Foo bar'))
+self.assertEqual('# Foo bar',
+ self.cct.putSpacesInLists('#Foo bar'))
+self.assertEqual('## Foo bar',
+ self.cct.putSpacesInLists('##Foo bar'))
+# right except the page is a redirect page
+self.assertEqual('# redirect',
+ self.cct.putSpacesInLists('#redirect'))
+self.assertEqual('#: Foo bar',
+ self.cct.putSpacesInLists('#:Foo bar'))
+self.assertEqual(':Foo bar',
+ self.cct.putSpacesInLists(':Foo bar'))
+self.assertEqual(':* Foo bar',
+ self.cct.putSpacesInLists(':*Foo bar'))
+
+def test_fixSyntaxSave(self):
+"""Test fixSyntaxSave method."""
+self.assertEqual(
+'[https://de.wikipedia.org]',
+self.cct.fixSyntaxSave('[[https://de.wikipedia.org]]'))
+self.assertEqual(
+'[https://de.wikipedia.org]',
+self.cct.fixSyntaxSave('[[https://de.wikipedia.org]'))
+self.assertEqual(
+'[https://de.wikipedia.org/w/api.php API]',
+self.cct.fixSyntaxSave('[https://de.wikipedia.org/w/api.php|API]'))
+
+def test_fixHtml(self):
+"""Test fixHtml method."""
+self.assertEqual("'''Foo''' bar",
+ self.cct.fixHtml('Foo bar'))
+self.assertEqual("Foo '''bar'''",
+ self.cct.fixHtml('Foo bar'))
+self.assertEqual("''Foo'' bar",
+ self.cct.fixHtml('Foo bar'))
+self.assertEqual("Foo ''bar''",
+ self.cct.fixHtml('Foo bar'))
+self.assertEqual('\n\n',
+ self.cct.fixHtml('\n\n'))
+self.assertEqual('\n=== Header ===\n',
+ self.cct.fixHtml('\nHeader\n'))
+
+def test_fixReferences(self):
+"""Test fixReferences method."""
+self.assertEqual('',
+ self.cct.fixReferences(''))
+self.assertEqual('bar',
+ self.cct.fixReferences('bar'))
+self.assertEqual('',
+ self.cct.fixReferences(''))
+self.assertEqual('',
+

[MediaWiki-commits] [Gerrit] [IMPROV] i18n: Deprecate non-mapping parameters - change (pywikibot/core)

2015-10-08 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244430

Change subject: [IMPROV] i18n: Deprecate non-mapping parameters
..

[IMPROV] i18n: Deprecate non-mapping parameters

Not using mapping parameters should be avoided when the text contains multiple
of them, as they might change order. And for consistency also translations
with only one parameter should use a mapping.

Change-Id: I60b32e4e415ed151458a484911a0cb222e2ab9c5
---
M pywikibot/i18n.py
1 file changed, 11 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/30/244430/1

diff --git a/pywikibot/i18n.py b/pywikibot/i18n.py
index ee2bb92..212906d 100644
--- a/pywikibot/i18n.py
+++ b/pywikibot/i18n.py
@@ -33,7 +33,7 @@
 import os
 import pkgutil
 
-from collections import defaultdict
+from collections import defaultdict, Mapping
 
 import pywikibot
 
@@ -41,6 +41,7 @@
 from pywikibot import Error
 from pywikibot import config
 from pywikibot.plural import plural_rules
+from pywikibot.tools import issue_deprecation_warning
 
 if sys.version_info[0] > 2:
 basestring = (str, )
@@ -448,6 +449,9 @@
 if parameters is None:
 return trans
 
+if not isinstance(parameters, Mapping):
+issue_deprecation_warning('parameters not being a mapping', None, 2)
+
 # else we check for PLURAL variants
 trans = _extract_plural(code, trans, parameters)
 if parameters:
@@ -470,7 +474,8 @@
 
 @param code: The language code
 @param twtitle: The TranslateWiki string title, in - format
-@param parameters: For passing parameters.
+@param parameters: For passing parameters. It should be a mapping but for
+backwards compatibility can also be a list, tuple or a single value.
 @param fallback: Try an alternate language code
 @type fallback: boolean
 """
@@ -512,6 +517,10 @@
 # send the language code back via the given list
 if code_needed:
 code.append(alt)
+
+if parameters is not None and not isinstance(parameters, Mapping):
+issue_deprecation_warning('parameters not being a Mapping', None, 2)
+
 if parameters:
 return trans % parameters
 else:

-- 
To view, visit https://gerrit.wikimedia.org/r/244430
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I60b32e4e415ed151458a484911a0cb222e2ab9c5
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] i18n: Deprecate twntranslate - change (pywikibot/core)

2015-10-08 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244431

Change subject: [IMPROV] i18n: Deprecate twntranslate
..

[IMPROV] i18n: Deprecate twntranslate

This deprecates `twntranslate` in favor of `twtranslate`. It also modifies
`_extract_plural` to require a mapping. `twntranslate` creates a dummy mapping
to simulate the deprecated types like lists to be used by `_extract_plural`.

This dummy mapping is also used by `translate` in order to deprecate the
non-mapping usage of parameters there. Not using a mapping for parameters in
i18n is only sensible if there is just one value but otherwise it might be that
parameters change the order in which case a list or tuple will produce
incorrect results. For the sake of consistency this deprecates all usage of
non-mapping parameters.

Change-Id: I5b97b10fbe0bab53993781c983db92c0aec30b30
---
M pywikibot/i18n.py
M scripts/archivebot.py
M scripts/category.py
M scripts/djvutext.py
M scripts/template.py
5 files changed, 130 insertions(+), 112 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/31/244431/1

diff --git a/pywikibot/i18n.py b/pywikibot/i18n.py
index 212906d..b8cc2cf 100644
--- a/pywikibot/i18n.py
+++ b/pywikibot/i18n.py
@@ -26,7 +26,6 @@
 __version__ = '$Id$'
 #
 
-import sys
 import re
 import locale
 import json
@@ -41,10 +40,7 @@
 from pywikibot import Error
 from pywikibot import config
 from pywikibot.plural import plural_rules
-from pywikibot.tools import issue_deprecation_warning
-
-if sys.version_info[0] > 2:
-basestring = (str, )
+from pywikibot.tools import deprecated, issue_deprecation_warning, StringTypes
 
 PLURAL_PATTERN = r'{{PLURAL:(?:%\()?([^\)]*?)(?:\)d)?\|(.*?)}}'
 
@@ -346,22 +342,16 @@
 
 """
 plural_items = re.findall(PLURAL_PATTERN, message)
+assert isinstance(parameters, Mapping), \
+'parameters is not Mapping but {0}'.format(type(parameters))
 if plural_items:  # we found PLURAL patterns, process it
-if len(plural_items) > 1 and isinstance(parameters, (tuple, list)) and 
\
-   len(plural_items) != len(parameters):
-raise ValueError("Length of parameter does not match PLURAL "
- "occurrences.")
-i = 0
 for selector, variants in plural_items:
-if isinstance(parameters, dict):
-num = int(parameters[selector])
-elif isinstance(parameters, basestring):
-num = int(parameters)
-elif isinstance(parameters, (tuple, list)):
-num = int(parameters[i])
-i += 1
-else:
-num = parameters
+num = parameters[selector]
+if not isinstance(num, int):
+issue_deprecation_warning('value {0} for {1} is not int but '
+  '{2}'.format(num, selector, 
type(num)),
+  None, 0)
+num = int(num)
 # TODO: check against plural_rules[code]['nplurals']
 try:
 index = plural_rules[code]['plural'](num)
@@ -392,14 +382,14 @@
 not be always the same. When fallback is iterable it'll return None if no
 code applies (instead of returning one).
 
-For PLURAL support have a look at the twntranslate method
+For PLURAL support have a look at the twtranslate method
 
 @param code: The language code
 @type code: string or Site object
 @param xdict: dictionary with language codes as keys or extended dictionary
   with family names as keys containing language dictionaries or
   a single (unicode) string. May contain PLURAL tags as
-  described in twntranslate
+  described in twtranslate
 @type xdict: dict, string, unicode
 @param parameters: For passing (plural) parameters
 @type parameters: dict, string, unicode, int
@@ -451,9 +441,12 @@
 
 if not isinstance(parameters, Mapping):
 issue_deprecation_warning('parameters not being a mapping', None, 2)
+plural_parameters = _PluralMappingAlias(parameters)
+else:
+plural_parameters = parameters
 
 # else we check for PLURAL variants
-trans = _extract_plural(code, trans, parameters)
+trans = _extract_plural(code, trans, plural_parameters)
 if parameters:
 try:
 return trans % parameters
@@ -463,21 +456,69 @@
 return trans
 
 
-def twtranslate(code, twtitle, parameters=None, fallback=True):
+def twtranslate(code, twtitle, parameters=None, fallback=True,
+only_plural=False):
 """
-Translate a message.
-
-The translations are retrieved from json files in messages_package_name.
+Translate a message using JSON files in messages_package_name.
 
 fallback parameter must be True for i18n and False for L10N or

[MediaWiki-commits] [Gerrit] [IMPROV] i18n_tests: Assert equality - change (pywikibot/core)

2015-10-08 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244621

Change subject: [IMPROV] i18n_tests: Assert equality
..

[IMPROV] i18n_tests: Assert equality

Instead of asserting an exception it could directly assert the result from
`twntranslate`. It's not testing that the %-operator is raising a `KeyError`
but it's testing that when not all parameters are applied it just applies
plural parameters.

Change-Id: Ifa02edeff1e70c278ec0011712135f538078a00a
---
M tests/i18n_tests.py
1 file changed, 4 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/21/244621/1

diff --git a/tests/i18n_tests.py b/tests/i18n_tests.py
index 3b5340d..bac92a0 100644
--- a/tests/i18n_tests.py
+++ b/tests/i18n_tests.py
@@ -304,10 +304,10 @@
 
 def testAllParametersExist(self):
 """Test that all parameters are required when using a dict."""
-with self.assertRaisesRegex(KeyError, repr(u'line')):
-# all parameters must be inside twntranslate
-i18n.twntranslate('de', 'test-multiple-plurals',
-  {'line': 1, 'page': 1}) % {'action': u'Ändere'}
+# all parameters must be inside twntranslate
+self.assertEqual(i18n.twntranslate('de', 'test-multiple-plurals',
+   {'line': 1, 'page': 1}),
+ 'Bot: %(action)s %(line)s Zeile von einer Seite.')
 
 def test_fallback_lang(self):
 """

-- 
To view, visit https://gerrit.wikimedia.org/r/244621
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifa02edeff1e70c278ec0011712135f538078a00a
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] cosmetic_changes: merge similar regexes - change (pywikibot/core)

2015-10-08 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244628

Change subject: [IMPROV] cosmetic_changes: merge similar regexes
..

[IMPROV] cosmetic_changes: merge similar regexes

Instead of having two pretty similar regexes it's in most cases possible to
combine them easily. This is also fixing “dash” into “pipe” in the comment
explaining how the external link fixer works.

It also adds the site parameter to each changed call.

Change-Id: I8513e34f873730ee9c92abb75c499f67cb3d86fc
---
M pywikibot/cosmetic_changes.py
1 file changed, 14 insertions(+), 26 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/28/244628/1

diff --git a/pywikibot/cosmetic_changes.py b/pywikibot/cosmetic_changes.py
index 169359f..ef8bfc3 100755
--- a/pywikibot/cosmetic_changes.py
+++ b/pywikibot/cosmetic_changes.py
@@ -706,17 +706,13 @@
 # 
r'\[https?://%s\.%s\.org/wiki/(?P\S+)\s+(?P.+?)\s?\]'
 # % (self.site.code, 
self.site.family.name),
 # r'[[\g|\g]]', exceptions)
-# external link in double brackets
+# external link in/starting with double brackets
 text = textlib.replaceExcept(
 text,
-r'\[\[(?Phttps?://[^\]]+?)\]\]',
-r'[\g]', exceptions)
-# external link starting with double bracket
-text = textlib.replaceExcept(text,
- r'\[\[(?Phttps?://.+?)\]',
- r'[\g]', exceptions)
-# external link and description separated by a dash, with
-# whitespace in front of the dash, so that it is clear that
+r'\[\[(?Phttps?://[^\]]+?)\]\]?',
+r'[\g]', exceptions, site=self.site)
+# external link and description separated by a pipe, with
+# whitespace in front of the pipe, so that it is clear that
 # the dash is not a legitimate part of the URL.
 text = textlib.replaceExcept(
 text,
@@ -739,14 +735,10 @@
 # Keep in mind that MediaWiki automatically converts  to 
 exceptions = ['nowiki', 'comment', 'math', 'pre', 'source',
   'startspace']
-text = textlib.replaceExcept(text, r'(?i)(.*?)', r"'''\1'''",
- exceptions)
-text = textlib.replaceExcept(text, r'(?i)(.*?)',
- r"'''\1'''", exceptions)
-text = textlib.replaceExcept(text, r'(?i)(.*?)', r"''\1''",
- exceptions)
-text = textlib.replaceExcept(text, r'(?i)(.*?)', r"''\1''",
- exceptions)
+text = textlib.replaceExcept(text, r'(?i)<(b|strong)>(.*?)',
+ r"'''\2'''", exceptions, site=self.site)
+text = textlib.replaceExcept(text, r'(?i)<(i|em)>(.*?)',
+ r"''\2''", exceptions, site=self.site)
 # horizontal line without attributes in a single line
 text = textlib.replaceExcept(text, r'(?i)([\r\n])([\r\n])',
  r'\1\2', exceptions)
@@ -797,19 +789,15 @@
 exceptions = ['nowiki', 'comment', 'math', 'pre', 'source',
   'startspace', 'gallery', 'hyperlink', 'interwiki', 
'link']
 # change  ccm ->  cm³
-text = textlib.replaceExcept(text, r'(\d)\s*ccm',
- r'\1' + u'cm³', exceptions)
-text = textlib.replaceExcept(text,
- r'(\d)\s*ccm', r'\1' + u'cm³',
- exceptions)
+text = textlib.replaceExcept(text, r'(\d)\s*(?:)?ccm',
+ r'\1cm³', exceptions,
+ site=self.site)
 # Solve wrong Nº sign with °C or °F
 # additional exception requested on fr-wiki for this stuff
 pattern = re.compile(u'«.*?»', re.UNICODE)
 exceptions.append(pattern)
-text = textlib.replaceExcept(text, r'(\d)\s*' + u'[º°]([CF])',
- r'\1' + u'°' + r'\2', exceptions)
-text = textlib.replaceExcept(text, r'(\d)\s*' + u'[º°]([CF])',
- r'\1' + u'°' + r'\2', exceptions)
+text = textlib.replaceExcept(text, r'(\d)\s*(?:)?[º°]([CF])',
+ r'\1°\2', exceptions, 
site=self.site)
 text = textlib.replaceExcept(text, u'º([CF])', u'°' + r'\1',
  exceptions)
 return text

-- 
To view, visit https://gerrit.wikimedia.org/r/244628
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8513e34f873730ee9c92abb75c499f67cb3d86fc
Gerrit-PatchSet: 1

[MediaWiki-commits] [Gerrit] [IMPROV] Iterate over the plural items - change (pywikibot/core)

2015-10-07 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/244360

Change subject: [IMPROV] Iterate over the plural items
..

[IMPROV] Iterate over the plural items

Instead of generating one list of plural items and to substitute always the
first occurrence it can utilize the matches generated by the regex and use that
to determine start and end point of the plural replacements.

This also converts the basestring parameter only once into an integer and it
does verify that `nplurals` is 1 when the `plurals` is not callable. And if
`nplurals` is 1 it verifies that that the value returned is 0.

It changes the `ValueError` into `TypeError` when the number of parameters
is to low as that is what %-notation will raise in such a case. It will also
complain about to many parameters in sequenced mode.

Change-Id: Id72f7bd55ddd816f6d15ba69e9b8e936333625fd
---
M pywikibot/i18n.py
M tests/i18n_tests.py
2 files changed, 50 insertions(+), 35 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/60/244360/1

diff --git a/pywikibot/i18n.py b/pywikibot/i18n.py
index ee2bb92..5299508 100644
--- a/pywikibot/i18n.py
+++ b/pywikibot/i18n.py
@@ -26,7 +26,6 @@
 __version__ = '$Id$'
 #
 
-import sys
 import re
 import locale
 import json
@@ -41,9 +40,7 @@
 from pywikibot import Error
 from pywikibot import config
 from pywikibot.plural import plural_rules
-
-if sys.version_info[0] > 2:
-basestring = (str, )
+from pywikibot.tools import StringTypes
 
 PLURAL_PATTERN = r'{{PLURAL:(?:%\()?([^\)]*?)(?:\)d)?\|(.*?)}}'
 
@@ -344,34 +341,48 @@
 @type parameters: int, basestring, tuple, list, dict
 
 """
-plural_items = re.findall(PLURAL_PATTERN, message)
-if plural_items:  # we found PLURAL patterns, process it
-if len(plural_items) > 1 and isinstance(parameters, (tuple, list)) and 
\
-   len(plural_items) != len(parameters):
-raise ValueError("Length of parameter does not match PLURAL "
- "occurrences.")
-i = 0
-for selector, variants in plural_items:
-if isinstance(parameters, dict):
-num = int(parameters[selector])
-elif isinstance(parameters, basestring):
-num = int(parameters)
-elif isinstance(parameters, (tuple, list)):
-num = int(parameters[i])
-i += 1
-else:
-num = parameters
-# TODO: check against plural_rules[code]['nplurals']
-try:
-index = plural_rules[code]['plural'](num)
-except KeyError:
-index = plural_rules['_default']['plural'](num)
-except TypeError:
-# we got an int, not a function
-index = plural_rules[code]['plural']
-repl = variants.split('|')[index]
-message = re.sub(PLURAL_PATTERN, repl, message, count=1)
-return message
+modified_message = ""
+end = 0
+i = 0
+if isinstance(parameters, StringTypes):
+parameters = int(parameters)
+
+for plural_match in re.finditer(PLURAL_PATTERN, message):
+if isinstance(parameters, dict):
+num = int(parameters[plural_match.group(1)])
+elif isinstance(parameters, (tuple, list)):
+if i >= len(parameters):
+raise TypeError('not enough arguments for format string')
+num = int(parameters[i])
+i += 1
+else:
+num = parameters
+
+try:
+rule = plural_rules[code]
+except KeyError:
+rule = plural_rules['_default']
+plural_value = rule['plural']
+if callable(plural_value):
+index = plural_value(num)
+else:
+assert rule['nplurals'] == 1
+index = plural_value
+if rule['nplurals'] == 1:
+assert index == 0
+
+plural_entries = plural_match.group(2).split('|')
+if index >= len(plural_entries):
+raise IndexError('requested plural {0} but only {1} '
+ 'provided'.format(index, len(plural_entries)))
+modified_message += message[end:plural_match.start()]
+modified_message += plural_entries[index]
+end = plural_match.end()
+
+if isinstance(parameters, (tuple, list)) and i < len(parameters):
+raise TypeError('not all arguments converted during string formatting')
+
+return modified_message + message[end:]
 
 
 DEFAULT_FALLBACK = ('_default', )
diff --git a/tests/i18n_tests.py b/tests/i18n_tests.py
index 3b5340d..9a81c82 100644
--- a/tests/i18n_tests.py
+++ b/tests/i18n_tests.py
@@ -287,13 +287,17 @@
 
 def testMultipleWrongParameterLength(self):
 """Test wrong parameter length."""
-err_msg = 'Length of parameter does not match PLURAL occurrences'
-with

[MediaWiki-commits] [Gerrit] [FIX] Remove guildwars test - change (pywikibot/core)

2015-10-05 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/243822

Change subject: [FIX] Remove guildwars test
..

[FIX] Remove guildwars test

The guildwars.wikia.com wiki has blocked the resource for the Travis tests.
This removes it for now so that the tests do not fail.

Bug: T114639
Change-Id: I36137e6556147241b46d3e85f027505c2026b67d
---
M tests/site_detect_tests.py
1 file changed, 0 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/22/243822/1

diff --git a/tests/site_detect_tests.py b/tests/site_detect_tests.py
index 2da5730..6a8ea4a 100644
--- a/tests/site_detect_tests.py
+++ b/tests/site_detect_tests.py
@@ -170,7 +170,6 @@
 def test_detect_site(self):
 """Test detection of MediaWiki sites."""
 self.assertSite('http://botwiki.sno.cc/wiki/$1')
-self.assertSite('http://guildwars.wikia.com/wiki/$1')
 self.assertSite('http://www.hrwiki.org/index.php/$1')  # v 1.15
 self.assertSite('http://www.proofwiki.org/wiki/$1')
 self.assertSite(

-- 
To view, visit https://gerrit.wikimedia.org/r/243822
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I36137e6556147241b46d3e85f027505c2026b67d
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] getLanguageLinks: Skip own site - change (pywikibot/core)

2015-10-05 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/243832

Change subject: [FIX] getLanguageLinks: Skip own site
..

[FIX] getLanguageLinks: Skip own site

When the text contains an langlink to its own site, it should be skipped as it
is not an interwiki link.

Conflicts:
tests/textlib_tests.py

Bug: T114608
Change-Id: I0856e198e817eac160b7f2682bffc9e65387ac6e
(cherry picked from commit c24650e8f55c74b78270cf66615433c702eac647)
---
M pywikibot/textlib.py
M tests/textlib_tests.py
2 files changed, 43 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/32/243832/1

diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index b40aba4..09281a4 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -482,7 +482,8 @@
 """
 Return a dict of inter-language links found in text.
 
-The returned dict uses language codes as keys and Page objects as values.
+The returned dict uses the site as keys and Page objects as values. It does
+not contain its own site.
 
 Do not call this routine directly, use Page.interwiki() method
 instead.
@@ -524,6 +525,9 @@
 pagetitle = pagetitle[:pagetitle.index('|')]
 # we want the actual page objects rather than the titles
 site = pywikibot.Site(code=lang, fam=fam)
+# skip language links to its own site
+if site == insite:
+continue
 try:
 result[site] = pywikibot.Page(site, pagetitle, insite=insite)
 except pywikibot.InvalidTitle:
diff --git a/tests/textlib_tests.py b/tests/textlib_tests.py
index 468d3f2..ba1c3ff 100644
--- a/tests/textlib_tests.py
+++ b/tests/textlib_tests.py
@@ -19,7 +19,9 @@
 from pywikibot import config
 from pywikibot.tools import OrderedDict
 
-from tests.aspects import unittest, TestCase, DefaultDrySiteTestCase
+from tests.aspects import (
+unittest, TestCase, DefaultDrySiteTestCase, SiteAttributeTestCase,
+)
 
 files = {}
 dirname = os.path.join(os.path.dirname(__file__), "pages")
@@ -541,6 +543,41 @@
  r'X\gX')
 
 
+class TestGetLanguageLinks(SiteAttributeTestCase):
+
+"""Test L{textlib.getLanguageLinks} function."""
+
+sites = {
+'enwp': {
+'family': 'wikipedia',
+'code': 'en',
+},
+'dewp': {
+'family': 'wikipedia',
+'code': 'de',
+},
+'commons': {
+'family': 'commons',
+'code': 'commons',
+},
+}
+
+example_text = '[[en:Site]] [[de:Site|Piped]] [[commons:Site]] 
[[baden:Site]]'
+
+@classmethod
+def setUpClass(cls):
+"""Define set of valid targets for the example text."""
+super(TestGetLanguageLinks, cls).setUpClass()
+cls.sites_set = set([cls.enwp, cls.dewp])
+
+def test_getLanguageLinks(self, key):
+"""Test if the function returns the correct titles and sites."""
+lang_links = textlib.getLanguageLinks(self.example_text, self.site)
+self.assertEqual(set(page.title() for page in lang_links.values()),
+ set(['Site']))
+self.assertEqual(set(lang_links), self.sites_set - set([self.site]))
+
+
 if __name__ == '__main__':
 try:
 unittest.main()

-- 
To view, visit https://gerrit.wikimedia.org/r/243832
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0856e198e817eac160b7f2682bffc9e65387ac6e
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Set name on generated test functions - change (pywikibot/core)

2015-10-05 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/243831

Change subject: Set __name__ on generated test functions
..

Set __name__ on generated test functions

When tests.aspects creates multiple tests for a specified list
of sites, it was setting the name of the generated test in the
test class dictionary, but nose 2.7 also needs the generated test
function to have __name__ set.

Change-Id: I5687ec91dba68ffa671641c6c219f6a20bc5a84e
(cherry picked from commit 4ec75cd7a3882f383fdfd0c3d6c6b2b13e839492)
---
M tests/aspects.py
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/31/243831/1

diff --git a/tests/aspects.py b/tests/aspects.py
index f96d9f0..4e0411c 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -782,6 +782,7 @@
 test_name = test + '_' + key
 
 dct[test_name] = wrap_method(key, sitedata, dct[test])
+dct[test_name].__name__ = str(test_name)
 
 if key in dct.get('expected_failures', []):
 dct[test_name] = unittest.expectedFailure(dct[test_name])

-- 
To view, visit https://gerrit.wikimedia.org/r/243831
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I5687ec91dba68ffa671641c6c219f6a20bc5a84e
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 
Gerrit-Reviewer: John Vandenberg 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] getLanguageLinks: Skip own site - change (pywikibot/core)

2015-10-05 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/243639

Change subject: [FIX] getLanguageLinks: Skip own site
..

[FIX] getLanguageLinks: Skip own site

When the text contains an langlink to its own site, it should be skipped as it
is not an interwiki link.

Bug: T114608
Change-Id: I0856e198e817eac160b7f2682bffc9e65387ac6e
---
M pywikibot/textlib.py
M tests/textlib_tests.py
2 files changed, 41 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/39/243639/1

diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index 2b6c529..d80ca1a 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -735,7 +735,8 @@
 """
 Return a dict of inter-language links found in text.
 
-The returned dict uses language codes as keys and Page objects as values.
+The returned dict uses the site as keys and Page objects as values. It does
+not contain its own site.
 
 Do not call this routine directly, use Page.interwiki() method
 instead.
@@ -777,6 +778,9 @@
 pagetitle = pagetitle[:pagetitle.index('|')]
 # we want the actual page objects rather than the titles
 site = pywikibot.Site(code=lang, fam=fam)
+# skip language links to its own site
+if site == insite:
+continue
 try:
 result[site] = pywikibot.Page(site, pagetitle)
 except pywikibot.InvalidTitle:
diff --git a/tests/textlib_tests.py b/tests/textlib_tests.py
index 5e84526..f450455 100644
--- a/tests/textlib_tests.py
+++ b/tests/textlib_tests.py
@@ -23,7 +23,7 @@
 
 from tests.aspects import (
 unittest, require_modules, TestCase, DefaultDrySiteTestCase,
-PatchingTestCase,
+PatchingTestCase, SiteAttributeTestCase,
 )
 
 files = {}
@@ -1187,6 +1187,41 @@
  r'X\gX')
 
 
+class TestGetLanguageLinks(SiteAttributeTestCase):
+
+"""Test L{textlib.getLanguageLinks} function."""
+
+sites = {
+'enwp': {
+'family': 'wikipedia',
+'code': 'en',
+},
+'dewp': {
+'family': 'wikipedia',
+'code': 'de',
+},
+'commons': {
+'family': 'commons',
+'code': 'commons',
+},
+}
+
+example_text = '[[en:Site]] [[de:Site|Piped]] [[commons:Site]] 
[[baden:Site]]'
+
+@classmethod
+def setUpClass(cls):
+"""Define set of valid targets for the example text."""
+super(TestGetLanguageLinks, cls).setUpClass()
+cls.sites_set = set([cls.enwp, cls.dewp])
+
+def test_getLanguageLinks(self, key):
+"""Test if the function returns the correct titles and sites."""
+lang_links = textlib.getLanguageLinks(self.example_text, self.site)
+self.assertEqual(set(page.title() for page in lang_links.values()),
+ set(['Site']))
+self.assertEqual(set(lang_links), self.sites_set - set([self.site]))
+
+
 if __name__ == '__main__':
 try:
 unittest.main()

-- 
To view, visit https://gerrit.wikimedia.org/r/243639
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I0856e198e817eac160b7f2682bffc9e65387ac6e
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Don't use shell argument - change (integration/jenkins)

2015-10-02 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/243140

Change subject: [IMPROV] Don't use shell argument
..

[IMPROV] Don't use shell argument

The `shell` argument in `subprocess.check_output` should be used very
carefully. But this is not necessary if we already split the arguments. This
now also uses a custom log format which will only print the commit message not
indented.

This also fixes various issues that it didn't detect lines which start with
"Change-Id" or "Bug" because it didn't consider the indentation.

Change-Id: I7724c6c75ed100ef11082e95f4f48ef3f5d91d6a
---
M tools/commit-message-validator.py
1 file changed, 6 insertions(+), 11 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/40/243140/1

diff --git a/tools/commit-message-validator.py 
b/tools/commit-message-validator.py
index 609518e..c8a532a 100755
--- a/tools/commit-message-validator.py
+++ b/tools/commit-message-validator.py
@@ -80,10 +80,8 @@
 changeid_line = False
 last_bug = False
 for lineno, line in enumerate(lines):
-# Strip leading spaces to remove `git log` indenting
-stripped = line.lstrip()
 rline = lineno + 1
-e = line_has_errors(lineno, stripped)
+e = line_has_errors(lineno, line)
 if e:
 errors.append(e)
 
@@ -123,7 +121,7 @@
 changeid_line = rline
 
 last_lineno = rline
-last_line = stripped
+last_line = line
 
 if last_lineno < 2:
 errors.append("Line %d: Expected at least 3 lines" % last_lineno)
@@ -149,14 +147,11 @@
 
 def main():
 """Validate the current HEAD commit message."""
-commit = subprocess.check_output('git log --pretty=raw -1', shell=True)
+commit = subprocess.check_output(['git', 'log', '--format=%B', 
'--no-color',
+  '-n1'])
 commit = commit.decode(locale.getpreferredencoding())
-lines = commit.splitlines()
-
-# Discard until the first blank line
-line = lines.pop(0)
-while line:
-line = lines.pop(0)
+# last line is always an empty line
+lines = commit.splitlines()[:-1]
 
 return check_message(lines)
 

-- 
To view, visit https://gerrit.wikimedia.org/r/243140
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7724c6c75ed100ef11082e95f4f48ef3f5d91d6a
Gerrit-PatchSet: 1
Gerrit-Project: integration/jenkins
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Support Python 3 in commit message validator - change (integration/jenkins)

2015-10-02 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/243131

Change subject: [IMPROV] Support Python 3 in commit message validator
..

[IMPROV] Support Python 3 in commit message validator

Even though the job will be probably run using Python 2 for a long time, the
issues using Python 3 are minor and easy to fix.

This does not fix the tests as the metaclass usage has been changed to much to
easily support both versions.

Change-Id: I52408ab450c929cdf10cddb1405ee0e4614d8473
---
M tools/commit-message-validator.py
1 file changed, 3 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/31/243131/1

diff --git a/tools/commit-message-validator.py 
b/tools/commit-message-validator.py
index 4852c8e..609518e 100755
--- a/tools/commit-message-validator.py
+++ b/tools/commit-message-validator.py
@@ -9,6 +9,7 @@
 https://www.mediawiki.org/wiki/Gerrit/Commit_message_guidelines
 """
 
+import locale
 import re
 import subprocess
 
@@ -141,7 +142,7 @@
 
 if errors:
 for e in errors:
-print e
+print(e)
 return 1
 return 0
 
@@ -149,6 +150,7 @@
 def main():
 """Validate the current HEAD commit message."""
 commit = subprocess.check_output('git log --pretty=raw -1', shell=True)
+commit = commit.decode(locale.getpreferredencoding())
 lines = commit.splitlines()
 
 # Discard until the first blank line

-- 
To view, visit https://gerrit.wikimedia.org/r/243131
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I52408ab450c929cdf10cddb1405ee0e4614d8473
Gerrit-PatchSet: 1
Gerrit-Project: integration/jenkins
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Allow Subject.originPage to be updated - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242518

Change subject: Allow Subject.originPage to be updated
..

Allow Subject.originPage to be updated

3488a4b1bb introduced a bug by changing a attribute to be a property
without a setter, however interwiki.py frequently sets this value.

Change-Id: I656e03a1d7499f7e1ba79185a8c55a5d3061681a
(cherry picked from commit 9011580e19c2aedfb0ba30c93daae04bd7d50151)
---
M pywikibot/interwiki_graph.py
1 file changed, 13 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/18/242518/1

diff --git a/pywikibot/interwiki_graph.py b/pywikibot/interwiki_graph.py
index 6f2c49a..c9e2918 100644
--- a/pywikibot/interwiki_graph.py
+++ b/pywikibot/interwiki_graph.py
@@ -91,6 +91,11 @@
 """Page on the origin wiki."""
 return self._origin
 
+@origin.setter
+def origin(self, value):
+"""Page on the origin wiki."""
+self._origin = value
+
 @property
 def originPage(self):
 """Deprecated property for the origin page.
@@ -100,6 +105,14 @@
 # TODO: deprecate this property
 return self.origin
 
+@originPage.setter
+def originPage(self, value):
+"""Deprecated property for the origin page.
+
+DEPRECATED.  Use origin.
+"""
+self.origin = value
+
 @property
 def foundIn(self):
 """Mapping of pages to others pages interwiki linked to it.

-- 
To view, visit https://gerrit.wikimedia.org/r/242518
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I656e03a1d7499f7e1ba79185a8c55a5d3061681a
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 
Gerrit-Reviewer: John Vandenberg 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] WikibasePage.id - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242517

Change subject: WikibasePage.id
..

WikibasePage.id

WikibasePage often failed to load due to the id attribute not
being set in the constructor, but being set in the subclasses.

Change-Id: I28253851498a2095e27ff9926eb03f4f294c6b49
(cherry picked from commit 40e45c8a8fa0e28f16c0622f83236b7fd967a38a)
---
M pywikibot/page.py
M tests/wikibase_tests.py
2 files changed, 33 insertions(+), 16 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/17/242517/1

diff --git a/pywikibot/page.py b/pywikibot/page.py
index 3f4f1be..d5cc6cf 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -3077,6 +3077,7 @@
 
 # .site forces a parse of the Link title to determine site
 self.repo = self.site
+self.id = self._link.title
 
 def _defined_by(self, singular=False):
 """
@@ -3460,7 +3461,7 @@
 # Special case for empty item.
 if title is None or title == '-1':
 super(ItemPage, self).__init__(site, u'-1', ns=ns)
-self.id = u'-1'
+assert self.id == '-1'
 return
 
 super(ItemPage, self).__init__(site, title, ns=ns)
@@ -3475,7 +3476,7 @@
 u"'%s' is not a valid item page title"
 % self._link.title)
 
-self.id = self._link.title
+assert self.id == self._link.title
 
 def title(self, **kwargs):
 """
@@ -3886,11 +3887,10 @@
 """
 WikibasePage.__init__(self, source, title,
   ns=source.property_namespace)
-Property.__init__(self, source, title)
-self.id = self.title(withNamespace=False).upper()
-if not self.id.startswith(u'P'):
+if not title or not self.id.startswith('P'):
 raise pywikibot.InvalidTitle(
 u"'%s' is not an property page title" % title)
+Property.__init__(self, source, self.id)
 
 def get(self, force=False, *args):
 """
diff --git a/tests/wikibase_tests.py b/tests/wikibase_tests.py
index 929dbd1..b7de738 100644
--- a/tests/wikibase_tests.py
+++ b/tests/wikibase_tests.py
@@ -727,27 +727,44 @@
 def test_wikibase_namespace_selection(self):
 """Test various ways to correctly specify the namespace."""
 wikidata = self.get_repo()
-page = pywikibot.page.ItemPage(wikidata, 'Q6')
-self.assertEqual(page.namespace(), 0)
-page = pywikibot.page.ItemPage(wikidata, title='Q6')
-self.assertEqual(page.namespace(), 0)
 
-page = pywikibot.page.WikibasePage(wikidata, title='Q6', ns=0)
+page = pywikibot.page.ItemPage(wikidata, 'Q60')
 self.assertEqual(page.namespace(), 0)
-page = pywikibot.page.WikibasePage(wikidata, title='Q6',
+page.get()
+
+page = pywikibot.page.ItemPage(wikidata, title='Q60')
+self.assertEqual(page.namespace(), 0)
+page.get()
+
+page = pywikibot.page.WikibasePage(wikidata, title='Q60', ns=0)
+self.assertEqual(page.namespace(), 0)
+page.get()
+
+page = pywikibot.page.WikibasePage(wikidata, title='Q60',
entity_type='item')
 self.assertEqual(page.namespace(), 0)
+page.get()
 
-page = pywikibot.page.PropertyPage(wikidata, 'Property:P60')
+page = pywikibot.page.PropertyPage(wikidata, 'Property:P6')
 self.assertEqual(page.namespace(), 120)
-page = pywikibot.page.PropertyPage(wikidata, 'P60')
-self.assertEqual(page.namespace(), 120)
+page.get()
 
-page = pywikibot.page.WikibasePage(wikidata, title='P60', ns=120)
+page = pywikibot.page.PropertyPage(wikidata, 'P6')
 self.assertEqual(page.namespace(), 120)
-page = pywikibot.page.WikibasePage(wikidata, title='P60',
+page.get()
+
+page = pywikibot.page.WikibasePage(wikidata, title='Property:P6')
+self.assertEqual(page.namespace(), 120)
+page.get()
+
+page = pywikibot.page.WikibasePage(wikidata, title='P6', ns=120)
+self.assertEqual(page.namespace(), 120)
+page.get()
+
+page = pywikibot.page.WikibasePage(wikidata, title='P6',
entity_type='property')
 self.assertEqual(page.namespace(), 120)
+page.get()
 
 def test_wrong_namespaces(self):
 """Test incorrect namespaces for Wikibase entities."""

-- 
To view, visit https://gerrit.wikimedia.org/r/242517
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I28253851498a2095e27ff9926eb03f4f294c6b49
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 
Gerrit-Reviewer: John Vandenberg

[MediaWiki-commits] [Gerrit] [IMRPOV] Deindent TestFilePage class content - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242519

Change subject: [IMRPOV] Deindent TestFilePage class content
..

[IMRPOV] Deindent TestFilePage class content

Change-Id: Icc0e21b14ba67906ea367b17003bf95b4856ddc3
(cherry picked from commit dee63962906335e9fdc976eb6a3919ad61218c35)
---
M tests/file_tests.py
1 file changed, 21 insertions(+), 21 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/19/242519/1

diff --git a/tests/file_tests.py b/tests/file_tests.py
index 625df91..cc861ed 100644
--- a/tests/file_tests.py
+++ b/tests/file_tests.py
@@ -121,33 +121,33 @@
 
 class TestFilePage(TestCase):
 
-"""Test FilePage.latest_revision_info.
+"""Test FilePage.latest_revision_info.
 
-These tests cover exceptions for all properties and methods
-in FilePage that rely on site.loadimageinfo.
+These tests cover exceptions for all properties and methods
+in FilePage that rely on site.loadimageinfo.
 
-"""
+"""
 
-family = 'wikipedia'
-code = 'test'
+family = 'wikipedia'
+code = 'test'
 
-cached = True
+cached = True
 
-def test_file_info_with_no_page(self):
-"""FilePage:latest_file_info raises NoPage for non existing 
pages."""
-site = self.get_site()
-image = pywikibot.FilePage(site, u'File:NoPage')
-self.assertFalse(image.exists())
-with self.assertRaises(pywikibot.NoPage):
-image = image.latest_file_info
+def test_file_info_with_no_page(self):
+"""FilePage:latest_file_info raises NoPage for non existing pages."""
+site = self.get_site()
+image = pywikibot.FilePage(site, u'File:NoPage')
+self.assertFalse(image.exists())
+with self.assertRaises(pywikibot.NoPage):
+image = image.latest_file_info
 
-def test_file_info_with_no_file(self):
-"""FilePage:latest_file_info raises PagerelatedError if no file is 
present."""
-site = self.get_site()
-image = pywikibot.FilePage(site, u'File:Test with no image')
-self.assertTrue(image.exists())
-with self.assertRaises(pywikibot.PageRelatedError):
-image = image.latest_file_info
+def test_file_info_with_no_file(self):
+"""FilePage:latest_file_info raises PagerelatedError if no file is 
present."""
+site = self.get_site()
+image = pywikibot.FilePage(site, u'File:Test with no image')
+self.assertTrue(image.exists())
+with self.assertRaises(pywikibot.PageRelatedError):
+image = image.latest_file_info
 
 
 if __name__ == '__main__':

-- 
To view, visit https://gerrit.wikimedia.org/r/242519
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Icc0e21b14ba67906ea367b17003bf95b4856ddc3
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] checkimages: Expect NoPage exception - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242520

Change subject: [FIX] checkimages: Expect NoPage exception
..

[FIX] checkimages: Expect NoPage exception

In compat the script called `getHash` which returned None if the image is
missing while `latest_file_info` raises a NoPage exception.

The script was previously fixed in 88ad247f and the usage was changed into
`getFileSHA1Sum` which would raise a `TypeError` if the page doesn't exist.

Bug: T105727
Change-Id: I9c39c41b86125ab7eeb1e842faad2a2cecbcb637
(cherry picked from commit 4bd04db59ba76534c165f2d674f5bebd1c156cd2)
---
M scripts/checkimages.py
1 file changed, 3 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/20/242520/1

diff --git a/scripts/checkimages.py b/scripts/checkimages.py
index e889f4d..985c554 100755
--- a/scripts/checkimages.py
+++ b/scripts/checkimages.py
@@ -905,8 +905,9 @@
 """Checking if the file is on commons."""
 pywikibot.output(u'Checking if [[%s]] is on commons...'
  % self.imageName)
-hash_found = self.image.latest_file_info.sha1
-if not hash_found:
+try:
+hash_found = self.image.latest_file_info.sha1
+except pywikibot.NoPage:
 return  # Image deleted, no hash found. Skip the image.
 
 site = pywikibot.Site('commons', 'commons')

-- 
To view, visit https://gerrit.wikimedia.org/r/242520
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9c39c41b86125ab7eeb1e842faad2a2cecbcb637
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Allow apostrophe in password - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242521

Change subject: [FIX] Allow apostrophe in password
..

[FIX] Allow apostrophe in password

If the password contains an apostrophe it'll mess up the line in the password
file. E.g. if the password is "foo'bar" it'll add the following line:

  ('username', 'foo'bar')

This escapes now any quotes and it'll generate the following line:

  ('username', 'foo\'bar')

Bug: T105783
Change-Id: I45501958362ef07e20b73448a11f11a4160f3136
(cherry picked from commit ac07c7a09ca43c66bb5352533ac88d766d64bdbf)
---
M .travis.yml
M tests/README.rst
2 files changed, 4 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/21/242521/1

diff --git a/.travis.yml b/.travis.yml
index 6135c7f..0ffea0d 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -48,7 +48,7 @@
   echo "usernames['wikipedia']['test'] = '$PYWIKIBOT2_USERNAME'" >> 
~/.pywikibot/user-config.py ;
   echo "usernames['wikidata']['test'] = '$PYWIKIBOT2_USERNAME'" >> 
~/.pywikibot/user-config.py ;
   echo "usernames['commons']['commons'] = '$PYWIKIBOT2_USERNAME'" >> 
~/.pywikibot/user-config.py ;
-  echo "('$PYWIKIBOT2_USERNAME', '$USER_PASSWORD')" > 
~/.pywikibot/passwordfile ;
+  printf "('%q', '%q')\n" "$PYWIKIBOT2_USERNAME" "$USER_PASSWORD" > 
~/.pywikibot/passwordfile ;
   echo "import os" >> ~/.pywikibot/user-config.py ;
   echo "password_file = os.path.expanduser('~/.pywikibot/passwordfile')" 
>> ~/.pywikibot/user-config.py ;
 fi
diff --git a/tests/README.rst b/tests/README.rst
index 3080680..2f9f91a 100644
--- a/tests/README.rst
+++ b/tests/README.rst
@@ -109,7 +109,9 @@
 4. The next build should run tests that require a logged in user
 
 While passwords in travis-ci environment variables are not leaked in normal
-operations, you are responsible for your own passwords.
+operations, you are responsible for your own passwords. If the password 
contains
+single quotes it is necessary to surround them in double quotes (see also
+`travis-ci #4350 `_).
 
 It is strongly recommended that an untrusted bot account is created for
 travis tests, using a password that is not shared with trusted accounts.

-- 
To view, visit https://gerrit.wikimedia.org/r/242521
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I45501958362ef07e20b73448a11f11a4160f3136
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Handle single quotes in usernames on Travis - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242522

Change subject: [FIX] Handle single quotes in usernames on Travis
..

[FIX] Handle single quotes in usernames on Travis

The username is written multiple times into the user-config surrounded by
single quotes. So if the username itself contains them these needs to be
escaped.

Bug: T105783
Change-Id: I54a7228f77e8cba653b25bd6e10bd4d4fee55e28
(cherry picked from commit b42724c107d62841a758a23f47d2e25da75a4052)
---
M .travis.yml
M tests/README.rst
2 files changed, 5 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/22/242522/1

diff --git a/.travis.yml b/.travis.yml
index 0ffea0d..d4de620 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -44,10 +44,10 @@
   - python -W error::UserWarning -m generate_user_files -dir:~/.pywikibot/ 
-family:$FAMILY -lang:$LANGUAGE -v -user:"$PYWIKIBOT2_USERNAME"
 
   - if [[ -n "$USER_PASSWORD" && -n "$PYWIKIBOT2_USERNAME" ]]; then
-  echo "usernames['wikipedia']['en'] = '$PYWIKIBOT2_USERNAME'" >> 
~/.pywikibot/user-config.py ;
-  echo "usernames['wikipedia']['test'] = '$PYWIKIBOT2_USERNAME'" >> 
~/.pywikibot/user-config.py ;
-  echo "usernames['wikidata']['test'] = '$PYWIKIBOT2_USERNAME'" >> 
~/.pywikibot/user-config.py ;
-  echo "usernames['commons']['commons'] = '$PYWIKIBOT2_USERNAME'" >> 
~/.pywikibot/user-config.py ;
+  printf "usernames['wikipedia']['en'] = '%q'\n" "$PYWIKIBOT2_USERNAME" >> 
~/.pywikibot/user-config.py ;
+  printf "usernames['wikipedia']['test'] = '%q'\n" "$PYWIKIBOT2_USERNAME" 
>> ~/.pywikibot/user-config.py ;
+  printf "usernames['wikidata']['test'] = '%q'\n" "$PYWIKIBOT2_USERNAME" 
>> ~/.pywikibot/user-config.py ;
+  printf "usernames['commons']['commons'] = '%q'\n" "$PYWIKIBOT2_USERNAME" 
>> ~/.pywikibot/user-config.py ;
   printf "('%q', '%q')\n" "$PYWIKIBOT2_USERNAME" "$USER_PASSWORD" > 
~/.pywikibot/passwordfile ;
   echo "import os" >> ~/.pywikibot/user-config.py ;
   echo "password_file = os.path.expanduser('~/.pywikibot/passwordfile')" 
>> ~/.pywikibot/user-config.py ;
diff --git a/tests/README.rst b/tests/README.rst
index 2f9f91a..b566cf4 100644
--- a/tests/README.rst
+++ b/tests/README.rst
@@ -109,7 +109,7 @@
 4. The next build should run tests that require a logged in user
 
 While passwords in travis-ci environment variables are not leaked in normal
-operations, you are responsible for your own passwords. If the password 
contains
+operations, you are responsible for your own passwords. If the variables 
contain
 single quotes it is necessary to surround them in double quotes (see also
 `travis-ci #4350 `_).
 

-- 
To view, visit https://gerrit.wikimedia.org/r/242522
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I54a7228f77e8cba653b25bd6e10bd4d4fee55e28
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] aspects: Imply user on write tests - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242524

Change subject: [FIX] aspects: Imply user on write tests
..

[FIX] aspects: Imply user on write tests

At the moment all writes require a user to be logged in but tests may not
define user and in that case the write test might fail because it's not working
on a logged in user. This sets user to True if it hasn't been set and is a
write test. So in case someone writes a write test without requiring a logged
in user it's necessary to explicitly set write to False now.

Change-Id: I4ef37ed6d9b28b72c0a1aa6be447818408c7107f
(cherry picked from commit 6255530f833c536a60d48197b54af9c5206405c1)
---
M tests/aspects.py
1 file changed, 2 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/24/242524/1

diff --git a/tests/aspects.py b/tests/aspects.py
index f96d9f0..83ee88c 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -753,6 +753,8 @@
 bases = tuple([CheckHostnameMixin] + list(bases))
 
 if 'write' in dct and dct['write']:
+if 'user' not in dct:
+dct['user'] = True
 bases = tuple([SiteWriteMixin] + list(bases))
 
 if ('user' in dct and dct['user']) or ('sysop' in dct and 
dct['sysop']):

-- 
To view, visit https://gerrit.wikimedia.org/r/242524
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I4ef37ed6d9b28b72c0a1aa6be447818408c7107f
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] script_test: Simplify test for stderr - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242523

Change subject: [IMPROV] script_test: Simplify test for stderr
..

[IMPROV] script_test: Simplify test for stderr

If there is expected output for that script it should just check that. The
additional check if it's an autorun script are unnecessary. The same is true
for the reverse and if there is no expected output and it is not an autorun
script then it should do the checks related to that.

Change-Id: Ic40aee0b1975ad1f785f27bafd3e56bc09511ada
(cherry picked from commit 00eb20fb0b3c1e801cdc08a12f477472600dcb73)
---
M tests/script_tests.py
1 file changed, 10 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/23/242523/1

diff --git a/tests/script_tests.py b/tests/script_tests.py
index bf11a8b..d676941 100644
--- a/tests/script_tests.py
+++ b/tests/script_tests.py
@@ -265,21 +265,18 @@
 if result['exit_code'] == -9:
 print(' killed', end='  ')
 
-if '-help' in args or error or \
-script_name not in auto_run_script_list:
+if error:
+self.assertIn(error, result['stderr'])
 
-if error:
-self.assertIn(error, result['stderr'])
+exit_codes = [0, 1, 2, -9]
+elif not is_autorun:
+if stderr_other == ['']:
+stderr_other = None
+self.assertIsNone(stderr_other)
+self.assertIn('Global arguments available for all',
+  result['stdout'])
 
-exit_codes = [0, 1, 2, -9]
-else:
-if stderr_other == ['']:
-stderr_other = None
-self.assertIsNone(stderr_other)
-self.assertIn('Global arguments available for all',
-  result['stdout'])
-
-exit_codes = [0]
+exit_codes = [0]
 else:
 # auto-run
 exit_codes = [0, -9]

-- 
To view, visit https://gerrit.wikimedia.org/r/242523
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic40aee0b1975ad1f785f27bafd3e56bc09511ada
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Report nose version - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242525

Change subject: Report nose version
..

Report nose version

Change-Id: I088b49b1e747512359ac7ef84b13c35182cbaf9d
(cherry picked from commit 53218c1d8a06b966f9d095c4e7806fee2a3399a4)
---
M .travis.yml
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/25/242525/1

diff --git a/.travis.yml b/.travis.yml
index d4de620..c6cbaab 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -61,6 +61,7 @@
 
 script:
   - if [[ "$USE_NOSE" == "1" ]]; then
+  nosetests --version ;
   if [[ "$SITE_ONLY" == "1" ]]; then
 echo "Running site tests only code ${LANGUAGE} on family ${FAMILY}" ;
 python setup.py nosetests --tests tests --verbosity=2 -a 
"family=$FAMILY,code=$LANGUAGE" ;

-- 
To view, visit https://gerrit.wikimedia.org/r/242525
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I088b49b1e747512359ac7ef84b13c35182cbaf9d
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 
Gerrit-Reviewer: John Vandenberg 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] New Wikipedia site: azb - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242514

Change subject: New Wikipedia site: azb
..

New Wikipedia site: azb

Change-Id: I754d7a4407688918349c5baaec331bd6694f6740
(cherry picked from commit 8a6981560df25d70791b91d932b3f80e39d7c8b1)
---
M pywikibot/families/wikipedia_family.py
1 file changed, 1 insertion(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/14/242514/1

diff --git a/pywikibot/families/wikipedia_family.py 
b/pywikibot/families/wikipedia_family.py
index e726b71..7bd1d35 100644
--- a/pywikibot/families/wikipedia_family.py
+++ b/pywikibot/families/wikipedia_family.py
@@ -80,6 +80,7 @@
 'mai', 'pih', 'got', 'xh', 'bi', 'sm', 'ss', 'rn', 'ki', 'pnt',
 'bm', 'iu', 'ee', 'lg', 'ts', 'fj', 'ak', 'ik', 'st', 'sg', 'ff',
 'dz', 'ny', 'ch', 'ti', 've', 'ks', 'tum', 'cr', 'gom', 'lrc',
+'azb',
 ]
 
 # Sites we want to edit but not count as real languages

-- 
To view, visit https://gerrit.wikimedia.org/r/242514
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I754d7a4407688918349c5baaec331bd6694f6740
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 
Gerrit-Reviewer: John Vandenberg 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Indexes in str.format - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242504

Change subject: [FIX] Indexes in str.format
..

[FIX] Indexes in str.format

The tools module was using the wrong index.

Conflicts:
scripts/redirect.py

Change-Id: Icb17bdfc91e923f62120c1584daef683b49cef68
(cherry picked from commit 876582033552377b1244e878b6f7262c19738d09)
---
M pywikibot/tools/__init__.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/04/242504/1

diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py
index 3c783e9..00c7fc2 100644
--- a/pywikibot/tools/__init__.py
+++ b/pywikibot/tools/__init__.py
@@ -1167,7 +1167,7 @@
 depth = get_wrapper_depth(wrapper) + 1
 args, varargs, kwargs, _ = inspect.getargspec(wrapper.__wrapped__)
 if varargs is not None and kwargs is not None:
-raise ValueError(u'{1} may not have * or ** args.'.format(
+raise ValueError('{0} may not have * or ** args.'.format(
 name))
 deprecated = set(__kw) & set(arg_names)
 if len(__args) > len(args):

-- 
To view, visit https://gerrit.wikimedia.org/r/242504
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Icb17bdfc91e923f62120c1584daef683b49cef68
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] add_text: Fixed unicode_literals position - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242515

Change subject: [FIX] add_text: Fixed unicode_literals position
..

[FIX] add_text: Fixed unicode_literals position

In the add_text script, the location of unicode_literals in 1e54a7d6 was added
in the docstring and not the code after it.

Change-Id: Ia23a9abc40c82cc1e1468e902b3812ef347a81c2
(cherry picked from commit edc204252a4eaffcb7dd94e2a3e9dd2551841d80)
---
M scripts/add_text.py
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/15/242515/1

diff --git a/scripts/add_text.py b/scripts/add_text.py
index b4b6532..b749256 100755
--- a/scripts/add_text.py
+++ b/scripts/add_text.py
@@ -41,7 +41,6 @@
 # This is a script to add a template to the top of the pages with
 # category:catname
 # Warning! Put it in one line, otherwise it won't work correctly.
-from __future__ import unicode_literals
 
 python add_text.py -cat:catname -summary:"Bot: Adding a template"
 -text:"{{Something}}" -except:"\{\{([Tt]emplate:|)[Ss]omething" -up
@@ -69,6 +68,8 @@
 #
 # Distributed under the terms of the MIT license.
 #
+from __future__ import unicode_literals
+
 __version__ = '$Id$'
 #
 

-- 
To view, visit https://gerrit.wikimedia.org/r/242515
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia23a9abc40c82cc1e1468e902b3812ef347a81c2
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] Release 2.0rc3 - change (pywikibot/core)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242526

Change subject: Release 2.0rc3
..

Release 2.0rc3

After all WMF wikis will be broken without d1854f5e a new version should be
released.

Change-Id: Ifef5202c4e60a638336c7eb2b3b0074962e4ff2c
---
M ChangeLog
M pywikibot/__init__.py
M setup.py
3 files changed, 12 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/26/242526/1

diff --git a/ChangeLog b/ChangeLog
index 7ba2f56..c0deb8a 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,13 @@
+Release 2.0rc3 (30 September 2015)
+
+
+Bugfixes (core)
+---
+6406479 New Wikipedia site: azb
+b503a1e Indexes in str.format
+d1854f5 MediaWikiVersion: Accept new wmf style
+718e4a1 i18n: always follow master
+
 Release 2.0rc2 (9 July 2015)
 
 
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index 0aaa184..b390191 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -7,7 +7,7 @@
 #
 from __future__ import unicode_literals
 
-__release__ = '2.0rc2'
+__release__ = '2.0rc3'
 __version__ = '$Id$'
 
 import datetime
diff --git a/setup.py b/setup.py
index e597c12..f04f7bb 100644
--- a/setup.py
+++ b/setup.py
@@ -133,7 +133,7 @@
 from setuptools import setup, find_packages
 
 name = 'pywikibot'
-version = '2.0rc2'
+version = '2.0rc3'
 github_url = 'https://github.com/wikimedia/pywikibot-core'
 
 setup(

-- 
To view, visit https://gerrit.wikimedia.org/r/242526
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifef5202c4e60a638336c7eb2b3b0074962e4ff2c
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Use generic commit message validation tests - change (integration/jenkins)

2015-09-30 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242551

Change subject: [IMPROV] Use generic commit message validation tests
..

[IMPROV] Use generic commit message validation tests

Instead of manually adding each commit message test pair, it now searches
automatically for the pairs, so it's easier to add new ones.

Change-Id: Ie1a5163f1d007634a7f4636748aa17d69a3a91c7
---
M tests/test_commit-message-validator.py
1 file changed, 43 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/integration/jenkins 
refs/changes/51/242551/1

diff --git a/tests/test_commit-message-validator.py 
b/tests/test_commit-message-validator.py
index c45935c..0919a5d 100644
--- a/tests/test_commit-message-validator.py
+++ b/tests/test_commit-message-validator.py
@@ -2,6 +2,8 @@
 
 import imp
 import os
+import StringIO
+import sys
 import unittest
 
 cmv = imp.load_source(
@@ -10,29 +12,47 @@
 '/tools/commit-message-validator.py')
 
 
+class MetaValidator(type):
+
+"""
+Create test class which checks the output for a message.
+
+It searches through `tests/test_commit-message-validator/` and loads all
+file pairs where there is a `.msg` and `.out` file. If both exist it'll
+create a test for this pair where the `.msg` is the commit message and the
+`.out` is the expected output.
+"""
+
+def __new__(cls, name, bases, dct):
+def create_test_method(msg, expected):
+def test(self):
+saved_stdout = sys.stdout
+try:
+out = StringIO.StringIO()
+sys.stdout = out
+self.assertEqual(
+1 if expected else 0,
+cmv.check_message(msg.splitlines()))
+self.assertEqual(expected, out.getvalue())
+finally:
+sys.stdout = saved_stdout
+return test
+
+base_path = os.path.join(os.path.dirname(__file__),
+ 'test_commit-message-validator')
+for fn in os.listdir(base_path):
+test, _, extension = fn.rpartition('.')
+fn = os.path.join(base_path, test)
+if extension == 'msg' and os.path.isfile(fn + '.out'):
+with open(fn + '.msg') as msg:
+with open(fn + '.out') as out:
+dct['test_' + test] = create_test_method(
+msg.read(), out.read())
+return super(MetaValidator, cls).__new__(cls, name, bases, dct)
+
+
 class TestCommitMessageValidator(unittest.TestCase):
-def test_check_message_errors(self):
-path = os.path.dirname(__file__) + '/test_commit-message-validator/'
-with open(path + 'check_message_errors.msg') as msg:
-with open(path + 'check_message_errors.out') as out:
-self._test_check_message(msg.read(), 1, out.read())
 
-def test_check_message_ok(self):
-path = os.path.dirname(__file__) + '/test_commit-message-validator/'
-with open(path + 'check_message_ok.msg') as msg:
-with open(path + 'check_message_ok.out') as out:
-self._test_check_message(msg.read(), 0, out.read())
+"""Validate the commit messages using test files."""
 
-def _test_check_message(self, msg, expect_status, expect_out):
-import sys
-import StringIO
-saved_stdout = sys.stdout
-try:
-out = StringIO.StringIO()
-sys.stdout = out
-self.assertEqual(
-expect_status,
-cmv.check_message(msg.splitlines()))
-self.assertEqual(expect_out, out.getvalue())
-finally:
-sys.stdout = saved_stdout
+__metaclass__ = MetaValidator

-- 
To view, visit https://gerrit.wikimedia.org/r/242551
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie1a5163f1d007634a7f4636748aa17d69a3a91c7
Gerrit-PatchSet: 1
Gerrit-Project: integration/jenkins
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Don't stop iterators using StopIteration - change (pywikibot/core)

2015-09-29 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242409

Change subject: [FIX] Don't stop iterators using StopIteration
..

[FIX] Don't stop iterators using StopIteration

With PEP 479 [1] it has been discouraged that a generator is stopped using
`StopIteration`. Instead if should just return from the function or method.
With 3.7 it'll convert that `StopIteration` into `RuntimeError` while 3.5 and
3.6 only issue deprecation warnings.

[1]: https://www.python.org/dev/peps/pep-0479/

Bug: T106224
Change-Id: I2743962594bc099dfeabc6f88d3654b178fb024d
---
M pywikibot/pagegenerators.py
M pywikibot/tools/__init__.py
2 files changed, 7 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/09/242409/1

diff --git a/pywikibot/pagegenerators.py b/pywikibot/pagegenerators.py
index d9a4ead..2263ed5 100644
--- a/pywikibot/pagegenerators.py
+++ b/pywikibot/pagegenerators.py
@@ -1685,7 +1685,7 @@
 while True:
 page_count = min(len(generator), step)
 if not page_count:
-raise StopIteration
+return
 
 for page in PreloadingGenerator(generator, page_count):
 yield page
diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py
index ce07b8d..4ce740c 100644
--- a/pywikibot/tools/__init__.py
+++ b/pywikibot/tools/__init__.py
@@ -786,9 +786,12 @@
 add = container_setitem
 
 for item in iterable:
-if (key(item) if key else item) not in container:
-add(item)
-yield item
+try:
+if (key(item) if key else item) not in container:
+add(item)
+yield item
+except StopIteration:
+return
 
 
 class CombinedError(KeyError, IndexError):

-- 
To view, visit https://gerrit.wikimedia.org/r/242409
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2743962594bc099dfeabc6f88d3654b178fb024d
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] MediaWikiVersion: Accept new wmf style - change (pywikibot/core)

2015-09-29 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242373

Change subject: [FIX] MediaWikiVersion: Accept new wmf style
..

[FIX] MediaWikiVersion: Accept new wmf style

With 1.27 the wmf version style changed adding a dash and period:

* `MediaWiki 1.27.0-wmf.1`
* `MediaWiki 1.26wmf24`

Change-Id: I49a4f56c3a273e0315ea4fe7f755a954f64263e5
---
M pywikibot/tools/__init__.py
1 file changed, 4 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/73/242373/1

diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py
index 4655d91..f37128e 100644
--- a/pywikibot/tools/__init__.py
+++ b/pywikibot/tools/__init__.py
@@ -422,6 +422,7 @@
 """
 
 MEDIAWIKI_VERSION = re.compile(
+# r'^(\d+(?:\.\d+)+)(-?wmf\.?(\d+)|alpha|beta(\d+)|-?rc\.?(\d+)|.*)?$')
 r'^(\d+(?:\.\d+)+)(wmf(\d+)|alpha|beta(\d+)|-?rc\.?(\d+)|.*)?$')
 
 @classmethod
@@ -450,10 +451,9 @@
 elif version_match.group(2) == 'alpha':
 self._dev_version = (1, )
 else:
-assert 'wmf' not in version_match.group(2)
-assert 'alpha' not in version_match.group(2)
-assert 'beta' not in version_match.group(2)
-assert 'rc' not in version_match.group(2)
+for dont in ('wmf', 'alpha', 'beta', 'rc'):
+assert dont in version_match.group(2), \
+'Found "{0}" in "{1}"'.format(dont, version_match.group(2))
 if version_match.group(2):
 debug('Additional unused version part '
   '"{0}"'.format(version_match.group(2)),

-- 
To view, visit https://gerrit.wikimedia.org/r/242373
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I49a4f56c3a273e0315ea4fe7f755a954f64263e5
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] MediaWikiVersion: Accept new wmf style - change (pywikibot/core)

2015-09-29 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242390

Change subject: [FIX] MediaWikiVersion: Accept new wmf style
..

[FIX] MediaWikiVersion: Accept new wmf style

With 1.27 the wmf version style changed adding a dash and period:

* `MediaWiki 1.27.0-wmf.1`
* `MediaWiki 1.26wmf24`

Bug: T114189
Change-Id: I458b44821d415be8e916978c728dc79451df376e
---
M pywikibot/tools/__init__.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/90/242390/1

diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py
index 8d9c4ff..3c783e9 100644
--- a/pywikibot/tools/__init__.py
+++ b/pywikibot/tools/__init__.py
@@ -322,7 +322,7 @@
 Any other suffixes are considered invalid.
 """
 
-MEDIAWIKI_VERSION = 
re.compile(r'^(\d+(?:\.\d+)+)(wmf(\d+)|alpha|beta(\d+)|-?rc\.?(\d+))?$')
+MEDIAWIKI_VERSION = 
re.compile(r'^(\d+(?:\.\d+)+)(-?wmf\.?(\d+)|alpha|beta(\d+)|-?rc\.?(\d+))?$')
 
 def parse(self, vstring):
 """Parse version string."""

-- 
To view, visit https://gerrit.wikimedia.org/r/242390
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I458b44821d415be8e916978c728dc79451df376e
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: 2.0
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] MediaWikiVersion: Accept new wmf style - change (pywikibot/core)

2015-09-29 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/242397

Change subject: [FIX] MediaWikiVersion: Accept new wmf style
..

[FIX] MediaWikiVersion: Accept new wmf style

With 1.27 the wmf version style changed adding a dash and period:

* `MediaWiki 1.27.0-wmf.1`
* `MediaWiki 1.26wmf24`

This patch was manually backported from 0a6516e6 for the master branch.

Bug: T114189
Change-Id: I458b44821d415be8e916978c728dc79451df376e
---
M pywikibot/tools/__init__.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/97/242397/1

diff --git a/pywikibot/tools/__init__.py b/pywikibot/tools/__init__.py
index 8d9c4ff..3c783e9 100644
--- a/pywikibot/tools/__init__.py
+++ b/pywikibot/tools/__init__.py
@@ -322,7 +322,7 @@
 Any other suffixes are considered invalid.
 """
 
-MEDIAWIKI_VERSION = 
re.compile(r'^(\d+(?:\.\d+)+)(wmf(\d+)|alpha|beta(\d+)|-?rc\.?(\d+))?$')
+MEDIAWIKI_VERSION = 
re.compile(r'^(\d+(?:\.\d+)+)(-?wmf\.?(\d+)|alpha|beta(\d+)|-?rc\.?(\d+))?$')
 
 def parse(self, vstring):
 """Parse version string."""

-- 
To view, visit https://gerrit.wikimedia.org/r/242397
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I458b44821d415be8e916978c728dc79451df376e
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Expect failure instead of skipping it - change (pywikibot/core)

2015-09-28 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241662

Change subject: [IMPROV] Expect failure instead of skipping it
..

[IMPROV] Expect failure instead of skipping it

The `testWebCiteOlder` test in `tests.weblib_tests.TestWebCite` doesn't work
because WebCite implementation doesn't work anymore. The memento implementation
though does work so it shouldn't expect a failure there.

To avoid this, 544f3207 skipped that test but it could still be executed
manually and fail then. Expecting a failure it won't fail but require that.

Change-Id: I065f8ce0a63e710d22129d0f7632d6e3f4a46ed8
---
M tests/__init__.py
M tests/weblib_tests.py
M tests/weblinkchecker_tests.py
3 files changed, 6 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/62/241662/1

diff --git a/tests/__init__.py b/tests/__init__.py
index 631b7ce..3d4509f 100644
--- a/tests/__init__.py
+++ b/tests/__init__.py
@@ -153,9 +153,6 @@
 'site_detect': [
 'test_IWM',  # very slow and tests include unnecessary sites
 ],
-'weblib': [
-'testWebCiteOlder',  # fails. T110640
-],
 }
 
 
diff --git a/tests/weblib_tests.py b/tests/weblib_tests.py
index 13c5a13..3d8b42c 100644
--- a/tests/weblib_tests.py
+++ b/tests/weblib_tests.py
@@ -77,6 +77,7 @@
 self.assertOneDeprecation()
 return archivedversion
 
+@unittest.expectedFailure  # see T110640
 def testWebCiteOlder(self):
 """Test WebCite for https://google.com as of January 2013."""
 archivedversion = self._get_archive_url('https://google.com', 
'20130101')
diff --git a/tests/weblinkchecker_tests.py b/tests/weblinkchecker_tests.py
index e3682a7..a516170 100644
--- a/tests/weblinkchecker_tests.py
+++ b/tests/weblinkchecker_tests.py
@@ -54,6 +54,11 @@
 timegate_uri = 'http://timetravel.mementoweb.org/webcite/timegate/'
 hostname = timegate_uri
 
+def testWebCiteOlder(self):
+"""Test WebCite using memento."""
+# This method is here so that it isn't considered an expected failure
+super(WeblibTestMementoWebCite, self).testWebCiteOlder()
+
 
 class TestMementoWebCite(MementoTestCase):
 

-- 
To view, visit https://gerrit.wikimedia.org/r/241662
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I065f8ce0a63e710d22129d0f7632d6e3f4a46ed8
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Removing disabled parts without mwpfh - change (pywikibot/core)

2015-09-27 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241565

Change subject: [FIX] Removing disabled parts without mwpfh
..

[FIX] Removing disabled parts without mwpfh

When `mwparserfromhell` is not installed it won't remove disabled parts unless
that parameter is explicitly set to `True`. Before 13cd73de it was always set
to `True` when using the regex variant but after that patch it only changed the
default value of `None` when `mwparserfromhell` is used.

Change-Id: I255823fc574c9d03f8d9961350a3545f3bcea3fb
---
M pywikibot/textlib.py
M tests/textlib_tests.py
2 files changed, 49 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/65/241565/1

diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
index 5998ac0..36c6e81 100644
--- a/pywikibot/textlib.py
+++ b/pywikibot/textlib.py
@@ -1291,9 +1291,8 @@
 use_mwparserfromhell = (config.use_mwparserfromhell and
 not isinstance(mwparserfromhell, Exception))
 
-if use_mwparserfromhell:
-if remove_disabled_parts is None:
-remove_disabled_parts = False
+if remove_disabled_parts is None:
+remove_disabled_parts = not use_mwparserfromhell
 
 if remove_disabled_parts:
 text = removeDisabledParts(text)
diff --git a/tests/textlib_tests.py b/tests/textlib_tests.py
index e50d5ef..d736970 100644
--- a/tests/textlib_tests.py
+++ b/tests/textlib_tests.py
@@ -23,6 +23,7 @@
 
 from tests.aspects import (
 unittest, require_modules, TestCase, DefaultDrySiteTestCase,
+PatchingTestCase,
 )
 
 files = {}
@@ -557,6 +558,52 @@
 self.assertTrue(m.group(0).endswith('foo {{bar}}'))
 
 
+class TestGenericTemplateParams(PatchingTestCase):
+
+"""Test whether the generic function forwards the call correctly."""
+
+net = False
+
+@PatchingTestCase.patched(textlib, 'extract_templates_and_params_mwpfh')
+def extract_mwpfh(self, text, *args, **kwargs):
+"""Patched call to extract_templates_and_params_mwpfh."""
+self._text = text
+self._mwpfh = True
+
+@PatchingTestCase.patched(textlib, 'extract_templates_and_params_regex')
+def extract_regex(self, text, *args, **kwargs):
+"""Patched call to extract_templates_and_params_regex."""
+self._text = text
+self._mwpfh = False
+
+def test_removing_disabled_parts_regex(self):
+"""Test removing disabled parts when using the regex variant."""
+self.patch(config, 'use_mwparserfromhell', False)
+textlib.extract_templates_and_params('{{a}}', True)
+self.assertEqual(self._text, '{{a}}')
+self.assertFalse(self._mwpfh)
+textlib.extract_templates_and_params('{{a}}', False)
+self.assertEqual(self._text, '{{a}}')
+self.assertFalse(self._mwpfh)
+textlib.extract_templates_and_params('{{a}}')
+self.assertEqual(self._text, '{{a}}')
+self.assertFalse(self._mwpfh)
+
+@require_modules('mwparserfromhell')
+def test_removing_disabled_parts_mwpfh(self):
+"""Test removing disabled parts when using the mwpfh variant."""
+self.patch(config, 'use_mwparserfromhell', True)
+textlib.extract_templates_and_params('{{a}}', True)
+self.assertEqual(self._text, '{{a}}')
+self.assertTrue(self._mwpfh)
+textlib.extract_templates_and_params('{{a}}', False)
+self.assertEqual(self._text, '{{a}}')
+self.assertTrue(self._mwpfh)
+textlib.extract_templates_and_params('{{a}}')
+self.assertEqual(self._text, '{{a}}')
+self.assertTrue(self._mwpfh)
+
+
 class TestReplaceLinks(TestCase):
 
 """Test the replace_links function in textlib."""

-- 
To view, visit https://gerrit.wikimedia.org/r/241565
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I255823fc574c9d03f8d9961350a3545f3bcea3fb
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Remove disabled parts when extracting templates - change (pywikibot/core)

2015-09-27 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241566

Change subject: [FIX] Remove disabled parts when extracting templates
..

[FIX] Remove disabled parts when extracting templates

When extracting the templates and values it should remove disabled parts to
avoid that the template or parameter names cannot be found.

Bug: T113892
Change-Id: Id9070dc9eb3e267f86f3f6b280889433a36267a5
---
M pywikibot/page.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/66/241566/1

diff --git a/pywikibot/page.py b/pywikibot/page.py
index ec7430a..251216f 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -2032,7 +2032,7 @@
 # WARNING: may not return all templates used in particularly
 # intricate cases such as template substitution
 titles = list(t.title() for t in self.templates())
-templates = textlib.extract_templates_and_params(self.text)
+templates = textlib.extract_templates_and_params(self.text, True)
 # backwards-compatibility: convert the dict returned as the second
 # element into a list in the format used by old scripts
 result = []

-- 
To view, visit https://gerrit.wikimedia.org/r/241566
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id9070dc9eb3e267f86f3f6b280889433a36267a5
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] Add translation keys for listpages - change (pywikibot/i18n)

2015-09-26 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241282

Change subject: [FEAT] Add translation keys for listpages
..

[FEAT] Add translation keys for listpages

Change-Id: Iea9f25318fbc49013ae0e9958bbba3d4a05f0a9c
---
A listpages/de.json
A listpages/en.json
A listpages/qqq.json
3 files changed, 24 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/i18n 
refs/changes/82/241282/1

diff --git a/listpages/de.json b/listpages/de.json
new file mode 100644
index 000..33dd003
--- /dev/null
+++ b/listpages/de.json
@@ -0,0 +1,8 @@
+{
+   "@metadata": {
+   "authors": [
+   "Fabian Neundorf"
+   ]
+   },
+   "listpages-save-list": "Bot: Sichere die generierte Liste der Seiten"
+}
diff --git a/listpages/en.json b/listpages/en.json
new file mode 100644
index 000..d996cb1
--- /dev/null
+++ b/listpages/en.json
@@ -0,0 +1,8 @@
+{
+   "@metadata": {
+   "authors": [
+   "Fabian Neundorf"
+   ]
+   },
+   "listpages-save-list": "Bot: Save the generated list of pages"
+}
diff --git a/listpages/qqq.json b/listpages/qqq.json
new file mode 100644
index 000..f767104
--- /dev/null
+++ b/listpages/qqq.json
@@ -0,0 +1,8 @@
+{
+   "@metadata": {
+   "authors": [
+   "Fabian Neundorf"
+   ]
+   },
+   "listpages-save-list": "Edit summary when listpages saves the list."
+}

-- 
To view, visit https://gerrit.wikimedia.org/r/241282
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iea9f25318fbc49013ae0e9958bbba3d4a05f0a9c
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/i18n
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] listpages: Put the generated list on a wiki - change (pywikibot/core)

2015-09-26 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241283

Change subject: [FEAT] listpages: Put the generated list on a wiki
..

[FEAT] listpages: Put the generated list on a wiki

This adds options to put the generated list on a wiki.

Change-Id: I1293919e9e9187c47730c07a04216abf4d882d3f
---
M scripts/listpages.py
1 file changed, 37 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/83/241283/1

diff --git a/scripts/listpages.py b/scripts/listpages.py
index d2dc095..d519835 100755
--- a/scripts/listpages.py
+++ b/scripts/listpages.py
@@ -50,6 +50,15 @@
  valid python encoding: utf-8, etc.).
  If not specified, it defaults to config.textfile_encoding.
 
+-put:Save the list to the defined page of the wiki. By default it does not
+ overwrite an exisiting page.
+
+-overwriteOverwrite the page if it exists. Can only by applied with -put.
+
+-summary: The summary text when the page is written. If it's one word just
+  containing letters, dashes and underscores it uses that as a
+  translation key.
+
 
 Custom format can be applied to the following items extrapolated from a
 page object:
@@ -83,10 +92,11 @@
 __version__ = '$Id$'
 #
 
+import re
 import os
 
 import pywikibot
-from pywikibot import config2 as config
+from pywikibot import config2 as config, i18n
 from pywikibot.pagegenerators import GeneratorFactory, parameterHelp
 
 docuReplacements = {'': parameterHelp}
@@ -172,6 +182,9 @@
 page_get = False
 base_dir = None
 encoding = config.textfile_encoding
+page_target = None
+overwrite = False
+summary = 'listpages-save-list'
 
 # Process global args and prepare generator args parser
 local_args = pywikibot.handle_args(args)
@@ -191,6 +204,12 @@
 base_dir = arg.partition(':')[2] or '.'
 elif arg.startswith('-encode:'):
 encoding = arg.partition(':')[2]
+elif arg.startswith('-put:'):
+page_target = arg.partition(':')[2]
+elif arg.startswith('-overwrite'):
+overwrite = True
+elif arg.startswith('-summary:'):
+summary = arg.partition(':')[2]
 else:
 genFactory.handleArg(arg)
 
@@ -214,13 +233,26 @@
   % base_dir)
 base_dir = None
 
+if page_target:
+site = pywikibot.Site()
+page_target = pywikibot.Page(site, page_target)
+if not overwrite and page_target.exists():
+pywikibot.bot.suggest_help(
+additional_text='Page "{0}" already exists.'.format(
+page_target.title()))
+return False
+if re.match('^[a-z_-]+$', summary):
+summary = i18n.twtranslate(site, summary)
+
 gen = genFactory.getCombinedGenerator()
 if gen:
 i = 0
+output_list = []
 for i, page in enumerate(gen, start=1):
 if not notitle:
 page_fmt = Formatter(page, outputlang)
-pywikibot.stdout(page_fmt.output(num=i, fmt=fmt))
+output_list += [page_fmt.output(num=i, fmt=fmt)]
+pywikibot.stdout(output_list[-1])
 if page_get:
 try:
 pywikibot.output(page.text, toStdout=True)
@@ -232,6 +264,9 @@
 with open(filename, mode='wb') as f:
 f.write(page.text.encode(encoding))
 pywikibot.output(u"%i page(s) found" % i)
+if page_target:
+page_target.text = '\n'.join(output_list)
+page_target.save(summary=summary)
 return True
 else:
 pywikibot.bot.suggest_help(missing_generator=True)

-- 
To view, visit https://gerrit.wikimedia.org/r/241283
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I1293919e9e9187c47730c07a04216abf4d882d3f
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] script_tests: Remove extension from dependency - change (pywikibot/core)

2015-09-26 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241303

Change subject: [FIX] script_tests: Remove extension from dependency
..

[FIX] script_tests: Remove extension from dependency

When the `memento_client` dependency for `weblinkchecker` was introduced in
3bf9bfac, the script dependency accidentally includes a filename extension. So
when it later built the list of tested scripts, it wouldn't skip that script
if the dependency isn't available because it was listed under a different name.

Change-Id: I8510cfa2553d5d5ad7e04c633b60b0a7ed9687be
---
M tests/script_tests.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/03/241303/1

diff --git a/tests/script_tests.py b/tests/script_tests.py
index 3756ddd..6f7aac9 100644
--- a/tests/script_tests.py
+++ b/tests/script_tests.py
@@ -45,7 +45,7 @@
 'panoramiopicker': ['BeautifulSoup'],
 'states_redirect': ['pycountry'],
 'patrol': ['mwlib'],
-'weblinkchecker.py': ['memento_client'],
+'weblinkchecker': ['memento_client'],
 }
 
 if PYTHON_VERSION < (2, 7):

-- 
To view, visit https://gerrit.wikimedia.org/r/241303
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I8510cfa2553d5d5ad7e04c633b60b0a7ed9687be
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] patrol: Replace add_to_tuples with defaultdict - change (pywikibot/core)

2015-09-26 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241328

Change subject: [IMPROV] patrol: Replace add_to_tuples with defaultdict
..

[IMPROV] patrol: Replace add_to_tuples with defaultdict

Instead of using the static method it now just uses a `defaultdict`.

Change-Id: I7cb543a4f1077e4a76c2abeff95ae4dade0f
---
M scripts/patrol.py
1 file changed, 8 insertions(+), 16 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/28/241328/1

diff --git a/scripts/patrol.py b/scripts/patrol.py
index 0adacee..d82657c 100755
--- a/scripts/patrol.py
+++ b/scripts/patrol.py
@@ -48,6 +48,8 @@
 __version__ = '$Id$'
 import time
 
+from collections import defaultdict
+
 import mwparserfromhell
 
 import pywikibot
@@ -165,17 +167,6 @@
 raise
 pywikibot.error(u'%s' % e)
 
-@staticmethod
-def add_to_tuples(tuples, user, page):
-"""Update tuples 'user' key by adding page."""
-if pywikibot.config.verbose_output:
-pywikibot.output(u"Adding %s:%s" % (user, page.title()))
-
-if user in tuples:
-tuples[user].append(page)
-else:
-tuples[user] = [page]
-
 def in_list(self, pagelist, title):
 """Check if title present in pagelist."""
 if pywikibot.config.verbose_output:
@@ -209,7 +200,7 @@
 
 def parse_page_tuples(self, wikitext, user=None):
 """Parse page details apart from 'user:' for use."""
-tuples = {}
+tuples = defaultdict(list)
 
 current_user = False
 parsed = mwparserfromhell.parse(wikitext)
@@ -263,19 +254,20 @@
 if self.is_wikisource_author_page(page):
 if pywikibot.config.verbose_output:
 pywikibot.output(u'Whitelist author: %s' % 
page)
-author = LinkedPagesRule(page)
-self.add_to_tuples(tuples, current_user, author)
+page = LinkedPagesRule(page)
 else:
 if pywikibot.config.verbose_output:
 pywikibot.output(u'Whitelist page: %s' % page)
-self.add_to_tuples(tuples, current_user, page)
+if pywikibot.config.verbose_output:
+pywikibot.output('Adding 
{0}:{1}'.format(current_user, page))
+tuples[current_user].append(page)
 elif pywikibot.config.verbose_output:
 pywikibot.output(u'Discarding whitelist page for '
  u'another user: %s' % page)
 else:
 raise Exception(u'No user set for page %s' % page)
 
-return tuples
+return dict(tuples)
 
 def is_wikisource_author_page(self, title):
 """Initialise author_ns if site family is 'wikisource' else pass."""

-- 
To view, visit https://gerrit.wikimedia.org/r/241328
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7cb543a4f1077e4a76c2abeff95ae4dade0f
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] patrol_tests: Define dry author namespace - change (pywikibot/core)

2015-09-26 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241479

Change subject: [FIX] patrol_tests: Define dry author namespace
..

[FIX] patrol_tests: Define dry author namespace

The patrol tests were testing a feature which also uses a wikisource specific
feature and require a namespace for the given author namespace number. This
adds a namespace for that author namespace number.

Change-Id: I6b57acc4f8d1639ca0963b39171212ab28c9136c
---
M tests/utils.py
1 file changed, 9 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/79/241479/1

diff --git a/tests/utils.py b/tests/utils.py
index b4335be..b2488f9 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -382,7 +382,15 @@
 self._msgcache = {'*': 'dummy entry', 'hello': 'world'}
 
 def _build_namespaces(self):
-return Namespace.builtin_namespaces(case=self.siteinfo['case'])
+ns_dict = Namespace.builtin_namespaces(case=self.siteinfo['case'])
+if hasattr(self.family, 'authornamespaces'):
+assert len(self.family.authornamespaces[self.code]) <= 1
+if self.family.authornamespaces[self.code]:
+author_ns = self.family.authornamespaces[self.code][0]
+assert author_ns not in ns_dict
+ns_dict[author_ns] = Namespace(
+author_ns, 'Author', case=self.siteinfo['case'])
+return ns_dict
 
 @property
 def userinfo(self):

-- 
To view, visit https://gerrit.wikimedia.org/r/241479
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6b57acc4f8d1639ca0963b39171212ab28c9136c
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] site_detect_tests: Ignore SSLError - change (pywikibot/core)

2015-09-26 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241487

Change subject: [IMPROV] site_detect_tests: Ignore SSLError
..

[IMPROV] site_detect_tests: Ignore SSLError

After 6344827d ignores failures because of timeouts, it can also ignore
failures because of SSLErrors.

Change-Id: I35aec5ee7b7763fe6f16f528c90f685525509080
---
M tests/site_detect_tests.py
1 file changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/87/241487/1

diff --git a/tests/site_detect_tests.py b/tests/site_detect_tests.py
index 2da5730..c384df5 100644
--- a/tests/site_detect_tests.py
+++ b/tests/site_detect_tests.py
@@ -9,7 +9,7 @@
 
 __version__ = '$Id$'
 
-from requests.exceptions import Timeout
+from requests.exceptions import Timeout, SSLError
 
 from pywikibot.exceptions import ServerError
 from pywikibot.site_detect import MWSite
@@ -85,7 +85,7 @@
 self._urls_tested.add(url)
 try:
 site = MWSite(url)
-except (ServerError, Timeout) as e:
+except (ServerError, Timeout, SSLError) as e:
 self.skips[url] = e
 return
 except Exception as e:

-- 
To view, visit https://gerrit.wikimedia.org/r/241487
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I35aec5ee7b7763fe6f16f528c90f685525509080
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] Test Python 3.5 on Travis - change (pywikibot/core)

2015-09-25 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241147

Change subject: [FEAT] Test Python 3.5 on Travis
..

[FEAT] Test Python 3.5 on Travis

As Python 3.5 has been officially released we should test on this. There are
minor issues with deprecations as `inspect.getargspec` has been deprecated. But
apart from a deprecation message it still works and will only be broken with
Python 3.6.

Bug: T106209
Change-Id: Ic8256184d1c1dbf841def5ad3846155564e05b75
---
M .travis.yml
M tests/utils.py
2 files changed, 7 insertions(+), 23 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/47/241147/1

diff --git a/.travis.yml b/.travis.yml
index 01b04ad..24cad4a 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -8,6 +8,7 @@
   - '2.7'
   - '3.3'
   - '3.4'
+  - '3.5'
   - '2.6'
 
 # OSX builds do not yet support Python
@@ -109,29 +110,6 @@
   matrix:
 - LANGUAGE=en FAMILY=wikipedia PYWIKIBOT2_TEST_PROD_ONLY=1
 - LANGUAGE=zh FAMILY=wikisource PYSETUP_TEST_EXTRAS=1 
PYWIKIBOT2_TEST_PROD_ONLY=1 PYWIKIBOT2_TEST_NO_RC=1
-
-matrix:
-  include:
-- python: '2.7_with_system_site_packages'  # equivalent to virtualenv: 
system_site_packages: true
-  env: LANGUAGE=he FAMILY=wikivoyage SITE_ONLY=1
-- python: '2.7'
-  env: LANGUAGE=en FAMILY=wpbeta SITE_ONLY=1 
OAUTH_DOMAIN="en.wikipedia.beta.wmflabs.org"
-- python: '3.3'
-  env: LANGUAGE=zh FAMILY=wpbeta SITE_ONLY=1 
OAUTH_DOMAIN="zh.wikipedia.beta.wmflabs.org"
-- python: '3.4'
-  env: LANGUAGE=en FAMILY=wsbeta SITE_ONLY=1
-- python: '2.7'
-  env: LANGUAGE=wikia FAMILY=wikia PYWIKIBOT2_TEST_NO_RC=1
-- python: '3.3'
-  env: LANGUAGE=en FAMILY=musicbrainz SITE_ONLY=1
-- python: '3.4'
-  env: LANGUAGE=test FAMILY=wikipedia SITE_ONLY=1 
OAUTH_DOMAIN="test.wikipedia.org"
-- python: '3.4'
-  env: LANGUAGE=test FAMILY=wikidata SITE_ONLY=1
-- python: '3.4'
-  env: LANGUAGE=ar FAMILY=wiktionary PYWIKIBOT2_TEST_NO_RC=1
-- python: '2.6'
-  env: LANGUAGE=wikidata FAMILY=wikidata SITE_ONLY=1
 
 notifications:
   email:
diff --git a/tests/utils.py b/tests/utils.py
index fb2fafb..654b91b 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -227,6 +227,12 @@
 else:
 skip_lines -= 1
 
+# Avoid failures because getargspec hasn't been removed yet: 
T106209
+if PYTHON_VERSION >= (3, 5, 0):
+if str(entry.message) == ('inspect.getargspec() is deprecated, 
'
+  'use inspect.signature() instead'):
+return
+
 log.append(entry)
 
 log = super(WarningSourceSkipContextManager, self).__enter__()

-- 
To view, visit https://gerrit.wikimedia.org/r/241147
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic8256184d1c1dbf841def5ad3846155564e05b75
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Don't limit cryptography module on 2.6 - change (pywikibot/core)

2015-09-25 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241251

Change subject: [IMPROV] Don't limit cryptography module on 2.6
..

[IMPROV] Don't limit cryptography module on 2.6

The cryptography module does complain on Python 2.6 that it's outdated. So it's
using the newest version which doesn't issue that warning. But that could
compromise the security as it won't include security patches.

This is now not limiting that version anymore and installing the newest version
on Python 2.6 as well. By default the deprecation warning is hidden but if the
user still wishes to suppress this warning they can install version 0.8.2 or
before.

Change-Id: If5270a0b2214c2f3b124d50180a122bfeeadc3cb
---
M requests-requirements.txt
M requirements.txt
M setup.py
M tests/utils.py
4 files changed, 16 insertions(+), 13 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/51/241251/1

diff --git a/requests-requirements.txt b/requests-requirements.txt
index 2923433..778c9f6 100644
--- a/requests-requirements.txt
+++ b/requests-requirements.txt
@@ -1,9 +1,5 @@
 requests
 
-# Dependency of pyOpenSSL. Use specific version to avoid expected
-# DeprecationWarning
-cryptography<=0.8.2 ; python_version < '2.7'
-
 # requests security extra
 # Bug T105767 on Python 2.7 release 9+
 pyOpenSSL ; python_full_version < '2.7.9' or python_version >= '3'
diff --git a/requirements.txt b/requirements.txt
index c7db758..cad79bf 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -21,10 +21,6 @@
 # mandatory; see README.conversion.txt
 requests
 
-# Dependency of pyOpenSSL. Use specific version to avoid expected
-# DeprecationWarning
-cryptography<=0.8.2 ; python_version < '2.7'
-
 # requests security extra
 pyOpenSSL ; python_full_version < '2.7.9'
 ndg-httpsclient ; python_full_version < '2.7.9'
diff --git a/setup.py b/setup.py
index 2a2d63f..6a120b1 100644
--- a/setup.py
+++ b/setup.py
@@ -107,11 +107,6 @@
 
 if sys.version_info[0] == 2:
 if PY26:
-# requests security extra includes pyOpenSSL. cryptography is the
-# dependency of pyOpenSSL. 0.8.2 is the newest and compatible version
-# for Python 2.6, which won't raise unexpected DeprecationWarning.
-extra_deps['security'].append('cryptography<=0.8.2')
-
 script_deps['replicate_wiki.py'] = ['argparse']
 dependencies.append('future>=0.15.0')  # provides collections backports
 
diff --git a/tests/utils.py b/tests/utils.py
index 654b91b..69f2fae 100644
--- a/tests/utils.py
+++ b/tests/utils.py
@@ -44,6 +44,9 @@
 
 OSWIN32 = (sys.platform == 'win32')
 
+PYTHON_26_CRYPTO_WARN = ('Python 2.6 is no longer supported by the Python core 
'
+ 'team, please upgrade your Python.')
+
 
 class DrySiteNote(RuntimeWarning):
 
@@ -231,6 +234,12 @@
 if PYTHON_VERSION >= (3, 5, 0):
 if str(entry.message) == ('inspect.getargspec() is deprecated, 
'
   'use inspect.signature() instead'):
+return
+# Avoid failures because cryptography is mentioning Python 2.6
+# is outdated
+if PYTHON_VERSION < (2, 7):
+if (isinstance(entry, DeprecationWarning) and
+str(entry.message) == PYTHON_26_CRYPTO_WARN):
 return
 
 log.append(entry)
@@ -557,9 +566,16 @@
 """
 Execute a command and capture outputs.
 
+On Python 2.6 it adds an option to ignore the deprecation warning from
+the cryptography package after the first entry of the command parameter.
+
 @param command: executable to run and arguments to use
 @type command: list of unicode
 """
+if PYTHON_VERSION < (2, 7):
+command.insert(
+1, '-W 
ignore:{0}:DeprecationWarning'.format(PYTHON_26_CRYPTO_WARN))
+
 # Any environment variables added on Windows must be of type
 # str() on Python 2.
 env = os.environ.copy()

-- 
To view, visit https://gerrit.wikimedia.org/r/241251
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If5270a0b2214c2f3b124d50180a122bfeeadc3cb
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] patrol: Test PatrolBot.parse_page_tuples - change (pywikibot/core)

2015-09-25 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/241022

Change subject: [FEAT] patrol: Test PatrolBot.parse_page_tuples
..

[FEAT] patrol: Test PatrolBot.parse_page_tuples

This makes the `PatrolBot` class a `SingleSiteBot` class and allows to set a
site via the constructor. It also adds a basic test for
`PatrolBot.parse_page_tuples` and `PatrolBot.in_list`.

Change-Id: I2cfb34a10e40e84634cd1ee15c5879741abe26d9
---
M scripts/patrol.py
A tests/patrolbot_tests.py
2 files changed, 82 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/22/241022/1

diff --git a/scripts/patrol.py b/scripts/patrol.py
index 5cc61e0..e37907d 100755
--- a/scripts/patrol.py
+++ b/scripts/patrol.py
@@ -25,11 +25,14 @@
 from __future__ import absolute_import, unicode_literals
 
 __version__ = '$Id$'
-import pywikibot
-from pywikibot import pagegenerators, Bot
 import mwlib.uparser  # used to parse the whitelist
 import mwlib.parser  # used to parse the whitelist
 import time
+
+import pywikibot
+
+from pywikibot import pagegenerators
+from pywikibot.bot import SingleSiteBot
 
 _logger = 'patrol'
 
@@ -40,7 +43,7 @@
 }
 
 
-class PatrolBot(Bot):
+class PatrolBot(SingleSiteBot):
 
 """Bot marks the edits as patrolled based on info obtained by whitelist."""
 
@@ -49,7 +52,7 @@
 'en': u'patrol_whitelist',
 }
 
-def __init__(self, **kwargs):
+def __init__(self, site=True, **kwargs):
 """
 Constructor.
 
@@ -67,10 +70,9 @@
 'versionchecktime': 300,
 'autopatroluserns': False
 })
-super(PatrolBot, self).__init__(**kwargs)
+super(PatrolBot, self).__init__(site, **kwargs)
 self.recent_gen = True
 self.user = None
-self.site = pywikibot.Site()
 if self.getOption('whitelist'):
 self.whitelist_pagename = self.getOption('whitelist')
 else:
diff --git a/tests/patrolbot_tests.py b/tests/patrolbot_tests.py
new file mode 100644
index 000..f423387
--- /dev/null
+++ b/tests/patrolbot_tests.py
@@ -0,0 +1,74 @@
+# -*- coding: utf-8  -*-
+"""Tests for the patrol script."""
+#
+# (C) Pywikibot team, 2015
+#
+# Distributed under the terms of the MIT license.
+#
+from __future__ import absolute_import, unicode_literals
+
+__version__ = '$Id$'
+#
+from scripts import patrol
+
+from tests.aspects import require_modules, unittest, DefaultDrySiteTestCase
+
+DUMMY_PAGE_TUPLES = """
+This is some text above the entries:
+
+== Header ==
+* [[User:Test 1]]: [[Page 1]], [[Page 2]]
+* [[User:Test 2]]: [[Page 2]], [[Page 4]], [[Page 6]]
+
+== Others ==
+* [[User:Prefixed]]: [[Special:PrefixIndex/Page 1]], 
[[Special:PREFIXINDEX/Page 2]]
+
+== More test 1 ==
+* [[User:Test 1]]: [[Page 3]]
+"""
+
+
+@require_modules('mwlib')
+class TestPatrolBot(DefaultDrySiteTestCase):
+
+"""Test the PatrolBot class."""
+
+def setUp(self):
+"""Create a bot dummy instance."""
+super(TestPatrolBot, self).setUp()
+self.bot = patrol.PatrolBot(self.site)
+
+def test_parse_page_tuples(self):
+"""Test parsing the page tuples from a dummy text."""
+tuples = self.bot.parse_page_tuples(DUMMY_PAGE_TUPLES)
+for gen_user in (1, 2):
+user = 'Test {0}'.format(gen_user)
+self.assertIn(user, tuples)
+self.assertEqual(tuples[user], ['Page {0}'.format(i * gen_user)
+for i in range(1, 4)])
+self.assertIn('Prefixed', tuples)
+self.assertEqual(tuples['Prefixed'], ['Page 1', 'Page 2'])
+
+def test_in_list(self):
+"""Test the method which returns whether a page is in the list."""
+# Return the title if there is an exact match
+self.assertEqual(self.bot.in_list(['Foo', 'Foobar'], 'Foo'), 'Foo')
+self.assertEqual(self.bot.in_list(['Foo', 'Foobar'], 'Foobar'), 
'Foobar')
+
+# Return the first entry which starts with the title if there is no
+# exact match
+self.assertEqual(self.bot.in_list(['Foo', 'Foobar'], 'Foob'), 'Foo')
+self.assertEqual(self.bot.in_list(['Foo', 'Foobar'], 'Foobarz'), 'Foo')
+self.assertEqual(self.bot.in_list(['Foo', 'Foobar', 'Bar'], 'Barz'), 
'Bar')
+
+# '' returns .* if there is no exact match
+self.assertEqual(self.bot.in_list([''], 'Foo'), '.*')
+self.assertEqual(self.bot.in_list(['', 'Foobar'], 'Foo'), '.*')
+self.assertEqual(self.bot.in_list(['', 'Foo'], 'Foo'), 'Foo')
+
+
+if __name__ == '__main__':
+try:
+unittest.main()
+except SystemExit:
+pass

-- 
To view, visit https://gerrit.wikimedia.org/r/241022
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2cfb34a10e40e84634cd1ee15c5879741abe26d9
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core

[MediaWiki-commits] [Gerrit] [FEAT] tests: Use generic function to get path - change (pywikibot/core)

2015-09-24 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/240893

Change subject: [FEAT] tests: Use generic function to get path
..

[FEAT] tests: Use generic function to get path

To access tests file there are various constants like `_data_dir`. But in most
cases `os.path.join` is used on them so it's more sensible to supply a function
which takes the filename and returns the combined result. Except for the
`_root_dir` all instances of `_*_dir` have been replaced by their `*_path`
counterpart.

Change-Id: I409413a9a480f9bd149cb72a365e76cde7d8e3e7
---
M tests/__init__.py
M tests/cache_tests.py
M tests/data_ingestion_tests.py
M tests/djvu_tests.py
M tests/dry_api_tests.py
M tests/fixes_tests.py
M tests/http_tests.py
M tests/pagegenerators_tests.py
M tests/pwb_tests.py
M tests/reflinks_tests.py
M tests/replacebot_tests.py
M tests/script_tests.py
M tests/tools_tests.py
M tests/upload_tests.py
M tests/uploadbot_tests.py
M tests/xmlreader_tests.py
16 files changed, 83 insertions(+), 95 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/93/240893/1

diff --git a/tests/__init__.py b/tests/__init__.py
index 290914d..0ec4575 100644
--- a/tests/__init__.py
+++ b/tests/__init__.py
@@ -9,10 +9,11 @@
 
 __version__ = '$Id$'
 
+import functools
 import os
 import warnings
 
-__all__ = ('requests', '_cache_dir', 'TestRequest',
+__all__ = ('requests', 'TestRequest',
'patch_request', 'unpatch_request')
 
 # Verify that the unit tests have a base working environment:
@@ -39,14 +40,27 @@
 from pywikibot.data.api import Request as _original_Request
 from pywikibot.data.api import CachedRequest
 
-_tests_dir = os.path.split(__file__)[0]
-_cache_dir = os.path.join(_tests_dir, 'apicache')
-_data_dir = os.path.join(_tests_dir, 'data')
-_images_dir = os.path.join(_data_dir, 'images')
+_root_dir = os.path.split(os.path.split(__file__)[0])[0]
+
+
+def root_path(*names):
+"""Return a path relative to the test directory."""
+return os.path.join(_root_dir, *names)
+
+
+def path_func(base_func, subpath):
+"""Return a function returning a path relative to the given directory."""
+return functools.partial(base_func, subpath)
+
+tests_path = path_func(root_path, 'tests')
+cache_path = path_func(tests_path, 'apicache')
+data_path = path_func(tests_path, 'data')
+
+images_path = path_func(data_path, 'images')
+xml_data_path = path_func(data_path, 'xml')
 
 # Find the root directory of the checkout
-_root_dir = os.path.split(_tests_dir)[0]
-_pwb_py = os.path.join(_root_dir, 'pwb.py')
+_pwb_py = root_path('pwb.py')
 
 library_test_modules = [
 'python',
@@ -141,7 +155,7 @@
 
 def _unknown_test_modules():
 """List tests which are to be executed."""
-dir_list = os.listdir(_tests_dir)
+dir_list = os.listdir(tests_path())
 all_test_list = [name[0:-9] for name in dir_list  # strip '_tests.py'
  if name.endswith('_tests.py') and
  not name.startswith('_')]   # skip __init__.py and _*
@@ -219,7 +233,7 @@
 
 
 CachedRequest._get_cache_dir = classmethod(
-lambda cls, *args: cls._make_dir(_cache_dir))
+lambda cls, *args: cls._make_dir(cache_path()))
 
 
 # Travis-CI builds are set to retry twice, which aims to reduce the number
diff --git a/tests/cache_tests.py b/tests/cache_tests.py
index 70add5c..b9960a1 100644
--- a/tests/cache_tests.py
+++ b/tests/cache_tests.py
@@ -13,7 +13,7 @@
 from pywikibot.site import BaseSite
 import scripts.maintenance.cache as cache
 
-from tests import _cache_dir
+from tests import cache_path
 from tests.aspects import unittest, TestCase
 
 
@@ -37,7 +37,7 @@
 
 def test_cache(self):
 """Test the apicache by doing _check_cache_entry over each entry."""
-cache.process_entries(_cache_dir, self._check_cache_entry)
+cache.process_entries(cache_path(), self._check_cache_entry)
 
 
 if __name__ == '__main__':
diff --git a/tests/data_ingestion_tests.py b/tests/data_ingestion_tests.py
index ee745a9..3201ba1 100644
--- a/tests/data_ingestion_tests.py
+++ b/tests/data_ingestion_tests.py
@@ -10,9 +10,7 @@
 
 __version__ = '$Id$'
 
-import os
-from tests import _data_dir
-from tests import _images_dir
+from tests import data_path, images_path
 from tests.aspects import unittest, TestCase, ScriptMainTestCase
 from scripts import data_ingestion
 
@@ -46,7 +44,7 @@
 
 def test_downloadPhoto(self):
 """Test download from http://upload.wikimedia.org/."";
-with open(os.path.join(_images_dir, 'MP_sounds.png'), 'rb') as f:
+with open(images_path('MP_sounds.png'), 'rb') as f:
 self.assertEqual(f.read(), self.obj.downloadPhoto().read())
 
 def test_findDuplicateImages(self):
@@ -82,7 +80,7 @@
 def setUp(self):
 """Set up unit test."""
 super(TestCSVReader, self).setUp()
-with open(os.path.join(_data_dir, 'csv_ingestion.csv'))

[MediaWiki-commits] [Gerrit] [IMPROV] nowcommons: Query at most one users - change (pywikibot/core)

2015-09-23 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/240359

Change subject: [IMPROV] nowcommons: Query at most one users
..

[IMPROV] nowcommons: Query at most one users

To verify if a page is used it's only necessary to query one using page instead
of all.

Change-Id: I9bbb206ede1723c33410c3b50e05dae71ecc265f
---
M scripts/nowcommons.py
1 file changed, 3 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/59/240359/1

diff --git a/scripts/nowcommons.py b/scripts/nowcommons.py
index cfd9d0b..433dd9c 100755
--- a/scripts/nowcommons.py
+++ b/scripts/nowcommons.py
@@ -374,9 +374,9 @@
 bot.run()
 # If the image is used with the urlname the
 # previous function won't work
-if len(list(pywikibot.FilePage(self.site,
-
page.title()).usingPages())) > 0 and \
-
self.getOption('replaceloose'):
+is_used = bool(list(pywikibot.FilePage(
+self.site, 
page.title()).usingPages(total=1)))
+if is_used and self.getOption('replaceloose'):
 bot = ImageBot(
 pg.FileLinksGenerator(
 localImagePage),

-- 
To view, visit https://gerrit.wikimedia.org/r/240359
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I9bbb206ede1723c33410c3b50e05dae71ecc265f
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] Use newer version of unicodecsv again - change (pywikibot/core)

2015-09-23 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/240574

Change subject: [IMPROV] Use newer version of unicodecsv again
..

[IMPROV] Use newer version of unicodecsv again

Version 0.14.0 of `unicodecsv` is not compatible with Python 2.6 so that with
f4ebbf92 it installed 0.13.0 (the version before that) on Python 2.6 systems.
But with 0.14.1 the compatibility was restored so that we only need to skip
0.14.0 installs.

Bug: T113222
Change-Id: I46ee49a95ea0491130f29d21bbc59e1610c72adf
---
M requirements.txt
M setup.py
2 files changed, 2 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/74/240574/1

diff --git a/requirements.txt b/requirements.txt
index c7db758..827c7e0 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -45,7 +45,7 @@
 git+https://github.com/nlhepler/pydot#egg=pydot-1.0.29
 
 # wikistats.py and scripts
-unicodecsv<=0.13 ; python_version < '2.7'
+unicodecsv!=0.14.0 ; python_version < '2.7'
 unicodecsv ; python_version < '3' and python_version >= '2.7'
 
 # cosmetic_changes and scripts/isbn
diff --git a/setup.py b/setup.py
index 2a2d63f..65d1b13 100644
--- a/setup.py
+++ b/setup.py
@@ -40,7 +40,7 @@
 
 # the irc module has no Python 2.6 support since 10.0
 irc_dep = 'irc==8.9' if sys.version_info < (2, 7) else 'irc'
-csv_dep = 'unicodecsv<=0.13' if PYTHON_VERSION < (2, 7) else 'unicodecsv'
+csv_dep = 'unicodecsv!=0.14.0' if PYTHON_VERSION < (2, 7) else 'unicodecsv'
 
 extra_deps = {
 # Core library dependencies

-- 
To view, visit https://gerrit.wikimedia.org/r/240574
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I46ee49a95ea0491130f29d21bbc59e1610c72adf
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] color_format: Only handle unicode strings - change (pywikibot/core)

2015-09-22 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/240291

Change subject: [FIX] color_format: Only handle unicode strings
..

[FIX] color_format: Only handle unicode strings

When using `unicode.format` it does return a `unicode` instance, but
`color_format` is based on Python's `Formatter` class which doesn't apply that
rule.

So it needs to workaround the issue that Python's `Formatter` will return the
`bytes` representation of an object when there is no further specification
(aka nothing after the colon). The default implementation will also return
`bytes` when there is no literal text and it might cause `UnicodeDecodeError`s
when there is literal text and the bytes representation does use non-ASCII
characters.

Bug: T113411
Change-Id: I0f8e0f1ac3e64c57918d99e9716c486270a87766
---
M pywikibot/tools/formatter.py
M tests/tools_formatter_tests.py
2 files changed, 65 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/91/240291/1

diff --git a/pywikibot/tools/formatter.py b/pywikibot/tools/formatter.py
index 151e4d9..1fdb170 100644
--- a/pywikibot/tools/formatter.py
+++ b/pywikibot/tools/formatter.py
@@ -15,6 +15,7 @@
 from string import Formatter
 
 from pywikibot.logging import output
+from pywikibot.tools import UnicodeType
 from pywikibot.userinterfaces.terminal_interface_base import colors
 
 
@@ -103,11 +104,39 @@
 if previous_literal:
 yield previous_literal, None, None, None
 
+def _vformat(self, *args, **kwargs):
+"""
+Override original `_vformat` to prevent that it changes into `bytes`.
+
+The original `_vformat` is returning `bytes` under certain
+curcumstances. It happens when the `format_string` is empty, when there
+is no literal text around it or when the field value is not a `unicode`
+already.
+"""
+result = super(_ColorFormatter, self)._vformat(*args, **kwargs)
+if not isinstance(result, UnicodeType):
+assert result == b''
+result = ''  # This is changing it into a unicode
+return result
+
 def vformat(self, format_string, args, kwargs):
-"""Return the normal format result but verify no colors are 
keywords."""
+"""
+Return the normal format result but verify no colors are keywords.
+
+@param format_string: The format template string
+@type format_string: unicode
+@param args: The positional field values
+@type args: sequence
+@param kwargs: The named field values
+@type kwargs: dict
+@return: The formatted string
+@rtype: unicode
+"""
 if self.colors.intersection(kwargs):  # kwargs use colors
 raise ValueError('Keyword argument(s) use valid color(s): ' +
  '", "'.join(self.colors.intersection(kwargs)))
+if not isinstance(format_string, UnicodeType):
+raise TypeError('expected str, got 
{0}'.format(type(format_string)))
 return super(_ColorFormatter, self).vformat(format_string, args,
 kwargs)
 
@@ -118,5 +147,8 @@
 
 It is automatically adding \03 in front of color fields so it's
 unnecessary to add them manually. Any other \03 in the text is disallowed.
+
+@param text: The format template string
+@type text: unicode
 """
 return _ColorFormatter().format(text, *args, **kwargs)
diff --git a/tests/tools_formatter_tests.py b/tests/tools_formatter_tests.py
index a441e35..bd3a2df 100644
--- a/tests/tools_formatter_tests.py
+++ b/tests/tools_formatter_tests.py
@@ -10,6 +10,7 @@
 __version__ = '$Id$'
 #
 from pywikibot.tools import formatter
+from pywikibot.tools import UnicodeMixin
 
 from tests.aspects import unittest, TestCase
 
@@ -35,20 +36,31 @@
 
 """Test color_format function in bot module."""
 
+class DummyUnicode(UnicodeMixin):
+
+def __unicode__(self):
+return 'ä'
+
 net = False
+
+def assert_format(self, format_string, expected, *args, **kwargs):
+"""Assert that color_format returns the expected string and type."""
+result = formatter.color_format(format_string, *args, **kwargs)
+self.assertEqual(result, expected)
+self.assertIsInstance(result, type(expected))
 
 def test_no_colors(self):
 """Test without colors in template string."""
-self.assertEqual(formatter.color_format('42'), '42')
-self.assertEqual(formatter.color_format('{0}', 42), '42')
-self.assertEqual(formatter.color_format('{ans}', ans=42), '42')
+self.assert_format('', '')
+self.assert_format('42', '42')
+self.assert_format('{0}', '42', 42)
+self.assert_format('before {0} after', 'before 42 after', 42)
+self.assert_format('{ans}', '42', ans=42)
 
 def

[MediaWiki-commits] [Gerrit] Correct Arabic writing - change (pywikibot/core)

2015-09-22 Thread XZise (Code Review)

XZise has submitted this change and it was merged.

Change subject: Correct Arabic writing
..


Correct Arabic writing

With patch fb9ca9fd some Arabic characters got changed. This is applying the
original wording from before the patch.

Change-Id: Id3699676d95accac68df5390f238774aba91e9a9
---
M scripts/lonelypages.py
1 file changed, 1 insertion(+), 1 deletion(-)

Approvals:
  XZise: Looks good to me, approved
  jenkins-bot: Verified



diff --git a/scripts/lonelypages.py b/scripts/lonelypages.py
index d8d8992..8097a48 100755
--- a/scripts/lonelypages.py
+++ b/scripts/lonelypages.py
@@ -91,7 +91,7 @@
 
 # The orphan template names in the different languages.
 _templates = {
-'ar': ('ﻲﺘﻴﻣﺓ', 'ﺕﺍﺮﻴﺧ={{ﻦﺴﺧ:ﺎﺴﻣ_ﺶﻫﺭ}} {{ﻦﺴﺧ:ﻉﺎﻣ}}'),
+'ar': ('يتيمة', 'تاريخ={{نسخ:اسم_شهر}} {{نسخ:عام}}'),
 'ca': ('Orfe', 'date={{subst:CURRENTMONTHNAME}} {{subst:CURRENTYEAR}}'),
 'en': ('Orphan', 'date={{subst:CURRENTMONTHNAME}} {{subst:CURRENTYEAR}}', 
['wi']),
 'it': ('O', '||mese={{subst:CURRENTMONTHNAME}} {{subst:CURRENTYEAR}}', 
['a']),

-- 
To view, visit https://gerrit.wikimedia.org/r/240077
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: Id3699676d95accac68df5390f238774aba91e9a9
Gerrit-PatchSet: 3
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Gerrit Patch Uploader 
Gerrit-Reviewer: Gerrit Patch Uploader 
Gerrit-Reviewer: John Vandenberg 
Gerrit-Reviewer: Ladsgroup 
Gerrit-Reviewer: XZise 
Gerrit-Reviewer: Zaher kadour 
Gerrit-Reviewer: jenkins-bot <>

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] wikistats_tests: Assert unicode - change (pywikibot/core)

2015-09-21 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239805

Change subject: [IMPROV] wikistats_tests: Assert unicode
..

[IMPROV] wikistats_tests: Assert unicode

Require that the returned total is a unicode type. This was separated from
6208baa3 as it changes the requirement of the test.

Change-Id: Ic7c48b868f2c3d6083b9157d6f1e8d1a033f0d81
---
M tests/wikistats_tests.py
1 file changed, 2 insertions(+), 6 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/05/239805/1

diff --git a/tests/wikistats_tests.py b/tests/wikistats_tests.py
index 883c146..4d59623 100644
--- a/tests/wikistats_tests.py
+++ b/tests/wikistats_tests.py
@@ -10,14 +10,10 @@
 __version__ = '$Id$'
 #
 
-import sys
-
 from pywikibot.data.wikistats import WikiStats, csv
+from pywikibot.tools import UnicodeType
 
 from tests.aspects import unittest, TestCase
-
-if sys.version_info[0] == 3:
-basestring = (str, )
 
 
 class WikiStatsTestCase(TestCase):
@@ -34,7 +30,7 @@
 self.assertIn('prefix', top)
 self.assertIn('total', top)
 self.assertEqual(top['prefix'], 'en')
-self.assertIsInstance(top['total'], basestring)
+self.assertIsInstance(top['total'], UnicodeType)
 self.assertEqual(ws.languages_by_size('wikipedia')[0], 'en')
 self.assertEqual(ws.languages_by_size('wikisource')[0], 'fr')
 

-- 
To view, visit https://gerrit.wikimedia.org/r/239805
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic7c48b868f2c3d6083b9157d6f1e8d1a033f0d81
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Use Python 2.6 compatible unicodecsv version - change (pywikibot/core)

2015-09-21 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239812

Change subject: [FIX] Use Python 2.6 compatible unicodecsv version
..

[FIX] Use Python 2.6 compatible unicodecsv version

The oldest version of unicodecsv which supports Python 2.6 is 0.13.0. This is
now forcing that not a newer version is installed when Python 2.6 is used.

Bug: T113222
Change-Id: I6dc6608e23770e833cfe439f94912eecf26a0e17
---
M setup.py
1 file changed, 2 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/12/239812/1

diff --git a/setup.py b/setup.py
index 8fe7a0e..7ce3859 100644
--- a/setup.py
+++ b/setup.py
@@ -40,6 +40,7 @@
 
 # the irc module has no Python 2.6 support since 10.0
 irc_dep = 'irc==8.9' if sys.version_info < (2, 7) else 'irc'
+csv_dep = 'unicodecsv<=0.13' if sys.version_info < (2, 7) else 'unicodecsv'
 
 extra_deps = {
 # Core library dependencies
@@ -59,7 +60,7 @@
 if PY2:
 # Additional core library dependencies which are only available on Python 2
 extra_deps.update({
-'csv': ['unicodecsv'],
+'csv': [csv_dep],
 'MySQL': ['oursql'],
 'unicode7': ['unicodedata2>=7.0.0-2'],
 })

-- 
To view, visit https://gerrit.wikimedia.org/r/239812
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6dc6608e23770e833cfe439f94912eecf26a0e17
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] APISite: Don't query unused inprop - change (pywikibot/core)

2015-09-20 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239665

Change subject: [IMPROV] APISite: Don't query unused inprop
..

[IMPROV] APISite: Don't query unused inprop

The `inprop` entries for `subjectid` and `talkid` were nowhere used so we don't
actually need to query them.

Change-Id: I2b2482973aa18451693e0f92523bebd38661b82d
---
M pywikibot/site.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/65/239665/1

diff --git a/pywikibot/site.py b/pywikibot/site.py
index dab70e5..8c5e3ea 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -2914,7 +2914,7 @@
 query = self._simple_request(
 action='query',
 prop='info',
-inprop=['protection', 'talkid', 'subjectid'],
+inprop=['protection'],
 titles=title,
 redirects=True)
 result = query.submit()

-- 
To view, visit https://gerrit.wikimedia.org/r/239665
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I2b2482973aa18451693e0f92523bebd38661b82d
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [IMPROV] listpages: Create Formatter once - change (pywikibot/core)

2015-09-20 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239682

Change subject: [IMPROV] listpages: Create Formatter once
..

[IMPROV] listpages: Create Formatter once

Instead of creating a `Formatter` instance on every page it's only creating
once, defining the format string and verifying that the output language is set.

It also makes `num` in `output` mandatory as most default format strings do not
work without defining it and the script is defining it always anyways. And if a
script is not needing it, it doesn't matter that it's supplied. The preset
format strings can now be references by an integer. The default value of
`output` was an int and wouldn't have worked anyway if the script hadn't
supplied it.

This is also using `color_format` so it's no longer necessary to prefix colors
using `\03` and escaping them in double curly brackets.

Change-Id: I2b145cb6cbc45ddb8b3dae38397302d0da8c7842
---
M scripts/listpages.py
1 file changed, 69 insertions(+), 52 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/82/239682/1

diff --git a/scripts/listpages.py b/scripts/listpages.py
index d2dc095..5bffc3e 100755
--- a/scripts/listpages.py
+++ b/scripts/listpages.py
@@ -10,13 +10,13 @@
 
 -format  Defines the output format.
 
- Can be a custom string according to python string.format() notation or
- can be selected by a number from following list (1 is default format):
+ Can be a custom string according to pywikibot.formatter.color_format()
+ notation or can be selected by a number from following (default 1):
  1 - u'{num:4d} {page.title}'
- --> 10 PageTitle
+ -->   10 PageTitle
 
  2 - u'{num:4d} {[[page.title]]}'
- --> 10 [[PageTitle]]
+ -->   10 [[PageTitle]]
 
  3 - u'{page.title}'
  --> PageTitle
@@ -24,17 +24,17 @@
  4 - u'{[[page.title]]}'
  --> [[PageTitle]]
 
- 5 - u'{num:4d} \03{{lightred}}{page.loc_title:<40}\03{{default}}'
- --> 10 PageTitle (colorised in lightred)
+ 5 - u'{num:4d} {lightred}{page.loc_title:<40}{default}'
+ -->   10 PageTitle (colorised in lightred)
 
  6 - u'{num:4d} {page.loc_title:<40} {page.can_title:<40}'
- --> 10 localised_Namespace:PageTitle canonical_Namespace:PageTitle
+ -->   10 localised_Namespace:PageTitle 
canonical_Namespace:PageTitle
 
  7 - u'{num:4d} {page.loc_title:<40} {page.trs_title:<40}'
- --> 10 localised_Namespace:PageTitle 
outputlang_Namespace:PageTitle
+ -->   10 localised_Namespace:PageTitle 
outputlang_Namespace:PageTitle
  (*) requires "outputlang:lang" set.
 
- num is the sequential number of the listed page.
+ num is the sequential number of the listed page starting from 1.
 
 -outputlang   Language for translation of namespaces.
 
@@ -52,7 +52,7 @@
 
 
 Custom format can be applied to the following items extrapolated from a
-page object:
+page object:
 
 site: obtained from page._link._site.
 
@@ -70,6 +70,9 @@
 trs_title: obtained from page._link.ns_title(onsite=onsite).
 If selected format requires trs_title, outputlang must be set.
 
+As the script is using L{pywikibot.tools.formatter.color_format}, it is not
+necessary to escape color fields using two curly brackets and it is not
+necessary to prefix them using C{\03}.
 
 
 """
@@ -88,33 +91,47 @@
 import pywikibot
 from pywikibot import config2 as config
 from pywikibot.pagegenerators import GeneratorFactory, parameterHelp
+from pywikibot.tools.formatter import color_format
 
 docuReplacements = {'': parameterHelp}
 
 
-class Formatter(object):
+class FormatEntry(object):
 
 """Structure with Page attributes exposed for formatting from cmd line."""
 
+def __init__(self, page, onsite=None, trs_title=None):
+"""Create the structure."""
+self.site = page._link.site
+self.title = page._link.title
+self.loc_title = page._link.canonical_title()
+self.can_title = page._link.ns_title()
+if onsite is not None:
+self.onsite = onsite
+if trs_title is not None:
+trs_title = trs_title
+
+
+class Formatter(object):
+
+"""Formatter handling each entry."""
+
 fmt_options = {
-'1': u"{num:4d} {page.title}",
-'2': u"{num:4d} [[{page.title}]]",
-'3': u"{page.title}",
-'4': u"[[{page.title}]]",
-'5': u"{num:4d} \03{{lightred}}{page.loc_title:<40}\03{{default}}",
-'6': u"{num:4d} {page.loc_title:<40} {page.can_title:<40}",
-'7': u"{num:4d} {page.loc_title:<40} {page.trs_title:<40}",
+1: u"{num:4d} {page.title}",
+2: u"{num:4d} [[{page.title}]]",
+3: u"{page.title}",
+4: u"[[{page.title}]]",
+5: u"{num:4d}

[MediaWiki-commits] [Gerrit] [FEAT] Support assertAPIError - change (pywikibot/core)

2015-09-20 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239683

Change subject: [FEAT] Support assertAPIError
..

[FEAT] Support assertAPIError

To assert a specific APIError the base class supports `assertAPIError` which
wraps around `assertRaises` and additionally checks if the error is of a
specific code and optionally a specific info. Like `assertRaises` it can be
used as a context manager when additional checks should be done afterwards.

Change-Id: Ib211974ec3acd17fb45c2280e084723257a88650
---
M tests/aspects.py
M tests/upload_tests.py
2 files changed, 50 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/83/239683/1

diff --git a/tests/aspects.py b/tests/aspects.py
index fa0e486..111aacb 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -45,7 +45,7 @@
 
 from pywikibot import Site
 from pywikibot.comms import http
-from pywikibot.data.api import Request as _original_Request
+from pywikibot.data.api import Request as _original_Request, APIError
 from pywikibot.exceptions import ServerError, NoUsername
 from pywikibot.family import WikimediaFamily
 from pywikibot.site import BaseSite
@@ -248,6 +248,54 @@
 
 assertPagelistTitles = assertPageTitlesEqual
 
+def assertAPIError(self, code, info=None, callable_obj=None, *args,
+   **kwargs):
+"""
+Assert that a specific APIError wrapped around L{assertRaises}.
+
+If no callable object is defined and it returns a context manager, that
+context manager will return the underlying context manager used by
+L{assertRaises}. So it's possible to access the APIError by using it's
+C{exception} attribute.
+
+@param code: The code of the error which must have happened.
+@type code: str
+@param info: The info string of the error or None if no it shouldn't be
+checked.
+@type info: str or None
+@param callable_obj: The object that will be tested. If None it returns
+a context manager like L{assertRaises}.
+@type callable_obj: callable
+@param args: The positional arguments forwarded to the callable object.
+@param kwargs: The keyword arguments forwared to the callable object.
+@return: The context manager if callable_obj is None and None 
otherwise.
+@rtype: None or context manager
+"""
+class AssertContext(object):
+def __enter__(ctx):
+ctx.cm = self.assertRaises(APIError, msg=kwargs.pop('msg', 
None))
+ctx.cm.__enter__()
+return ctx.cm
+
+def __exit__(ctx, exc_type, exc_value, tb):
+print('EXIT')
+result = ctx.cm.__exit__(exc_type, exc_value, tb)
+if isinstance(exc_value, APIError):
+assert result is True
+ctx.exception = exc_value
+self.assertEqual(exc_value.code, code)
+if info:
+self.assertEqual(exc_value, info)
+return result
+
+def handle(ctx):
+if callable_obj is None:
+return ctx
+with ctx:
+callable_obj(*args, **kwargs)
+
+return AssertContext().handle()
+
 
 class TestTimerMixin(TestCaseBase):
 
diff --git a/tests/upload_tests.py b/tests/upload_tests.py
index 610a9d8..1e4b787 100644
--- a/tests/upload_tests.py
+++ b/tests/upload_tests.py
@@ -17,8 +17,6 @@
 
 import pywikibot
 
-from pywikibot.data.api import APIError
-
 from tests import _images_dir
 from tests.aspects import unittest, TestCase
 
@@ -107,9 +105,8 @@
 self._finish_upload(chunk_size, self.sounds_png)
 
 # Check if it's still cached
-with self.assertRaises(APIError) as cm:
+with self.assertAPIError('siiinvalidsessiondata') as cm:
 self.site.stash_info(self._file_key)
-self.assertEqual(cm.exception.code, 'siiinvalidsessiondata')
 self.assertTrue(cm.exception.info.startswith('File not found'),
 'info ({0}) did not start with '
 '"File not found"'.format(cm.exception.info))

-- 
To view, visit https://gerrit.wikimedia.org/r/239683
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib211974ec3acd17fb45c2280e084723257a88650
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] win32_unicode: Adjust classes to IOBase - change (pywikibot/core)

2015-09-20 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239750

Change subject: [FIX] win32_unicode: Adjust classes to IOBase
..

[FIX] win32_unicode: Adjust classes to IOBase

With 72afc93e the `UnicodeOutput` class subclasses `IOBase` which already
handles `closed`, `close()` and `isatty()`.

Change-Id: Ifc78ba65198bf9add0bf239f5faba711366dd573
---
M pywikibot/userinterfaces/win32_unicode.py
1 file changed, 0 insertions(+), 10 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/50/239750/1

diff --git a/pywikibot/userinterfaces/win32_unicode.py 
b/pywikibot/userinterfaces/win32_unicode.py
index f86185b..74b60c3 100755
--- a/pywikibot/userinterfaces/win32_unicode.py
+++ b/pywikibot/userinterfaces/win32_unicode.py
@@ -96,21 +96,11 @@
 self._hConsole = hConsole
 self._stream = stream
 self._fileno = fileno
-self.closed = False
 self.softspace = False
 self.mode = 'w'
 self.encoding = 'utf-8'
 self.name = name
 self.flush()
-
-def isatty(self):
-"""Return whether it's a tty."""
-return False
-
-def close(self):
-"""Set the stream to be closed."""
-# don't really close the handle, that would only cause problems
-self.closed = True
 
 def fileno(self):
 """Return the fileno."""

-- 
To view, visit https://gerrit.wikimedia.org/r/239750
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifc78ba65198bf9add0bf239f5faba711366dd573
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [WIP][IMPROV] tests: Hostname check as decorator - change (pywikibot/core)

2015-09-20 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239757

Change subject: [WIP][IMPROV] tests: Hostname check as decorator
..

[WIP][IMPROV] tests: Hostname check as decorator

This implements the hostname checks as a decorator so it's not necessary to
subclass from `CheckHostnameMixin` implicitly.

Change-Id: I6401e11601bd1c714a4c5b038bb2c9de18371241
---
M tests/aspects.py
M tests/wikistats_tests.py
2 files changed, 76 insertions(+), 29 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/57/239757/1

diff --git a/tests/aspects.py b/tests/aspects.py
index fa0e486..7ef6a6d 100644
--- a/tests/aspects.py
+++ b/tests/aspects.py
@@ -274,23 +274,51 @@
 super(TestTimerMixin, self).tearDown()
 
 
-def require_modules(*required_modules):
-"""Require that the given list of modules can be imported."""
-def test_requirement(obj):
+class RequirementDecorator(object):
+
+"""Utility class to write decorators which require something."""
+
+@classmethod
+def decorator(cls, *args, **kwargs):
+"""Create new instance and return decorate method."""
+return cls(*args, **kwargs).decorate
+
+def decorate(self, obj):
 """Test the requirement and return an optionally decorated object."""
+reason = self.test_requirement()
+if reason:
+return unittest.skip(reason)(obj)
+else:
+return obj
+
+def test_requirement(self):
+"""Test whether the requirement is met and return the reason if not."""
+raise NotImplementedError()
+
+
+class CheckModules(RequirementDecorator):
+
+"""Require that the given list of modules can be imported."""
+
+def __init__(self, *modules):
+"""Create new decorator instance requiring the given modules."""
+self.required_modules = modules
+
+def test_requirement(self):
+"""Test that the required modules can be imported."""
 missing = []
-for required_module in required_modules:
+for required_module in self.required_modules:
 try:
 __import__(required_module, globals(), locals(), [], 0)
 except ImportError:
 missing += [required_module]
 if missing:
-return unittest.skip('{0} not installed'.format(
-', '.join(missing)))(obj)
+return '{0} not installed'.format(', '.join(missing))
 else:
-return obj
+return None
 
-return test_requirement
+
+require_modules = CheckModules.decorator
 
 
 class DisableSiteMixin(TestCaseBase):
@@ -417,8 +445,6 @@
 
 """Check the hostname is online before running tests."""
 
-_checked_hostnames = {}
-
 @classmethod
 def setUpClass(cls):
 """
@@ -431,21 +457,38 @@
 if not hasattr(cls, 'sites'):
 return
 
+hostnames = []
 for key, data in cls.sites.items():
 if 'hostname' not in data:
 raise Exception('%s: hostname not defined for %s'
 % (cls.__name__, key))
-hostname = data['hostname']
+hostnames += [data['hostname']]
 
-if hostname in cls._checked_hostnames:
-if isinstance(cls._checked_hostnames[hostname], Exception):
-raise unittest.SkipTest(
-'%s: hostname %s failed (cached): %s'
-% (cls.__name__, hostname,
-   cls._checked_hostnames[hostname]))
-elif cls._checked_hostnames[hostname] is False:
-raise unittest.SkipTest('%s: hostname %s failed (cached)'
-% (cls.__name__, hostname))
+reason = CheckHostname(*hostnames).test_requirement()
+if reason is not None:
+raise unittest.SkipTest('{0}: {1}'.format(cls.__name__, reason))
+
+
+class CheckHostname(RequirementDecorator):
+
+"""Require that the given list of hostnames can be accessed."""
+
+_checked_hostnames = {}
+
+def __init__(self, *hostnames):
+"""Create new instance with the given hostnames."""
+self.hostnames = hostnames
+
+def test_requirement(self):
+"""Iterate over each hostname and test their availability."""
+for hostname in self.hostnames:
+
+if hostname in self._checked_hostnames:
+if isinstance(self._checked_hostnames[hostname], Exception):
+return 'hostname {0} failed (cached): {1}'.format(
+hostname, self._checked_hostnames[hostname])
+elif self._checked_hostnames[hostname] is False:
+return 'hostname {0} failed (cached)'.format(hostname)
 else:
 continue
 
@@ -462,19 +505,22 @@
 if r.status not in

[MediaWiki-commits] [Gerrit] [FIX] config: Inverted PY2 logic - change (pywikibot/core)

2015-09-19 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239550

Change subject: [FIX] config: Inverted PY2 logic
..

[FIX] config: Inverted PY2 logic

In 97f69a60, `PY2` was redefined in the `config2` module instead of using
`tools`' implementation. This adds an error as the logic was actually detecting
//non-PY2//. To minimize the redundancy it's now using `tools`' implementation.

Change-Id: Id467fd3d0c176a70d9192639e694fd3b4fd2129a
---
M pywikibot/config2.py
1 file changed, 1 insertion(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/50/239550/1

diff --git a/pywikibot/config2.py b/pywikibot/config2.py
index b383cc3..c5ad2c1 100644
--- a/pywikibot/config2.py
+++ b/pywikibot/config2.py
@@ -54,9 +54,9 @@
 
 from pywikibot import __url__
 from pywikibot.logging import error, output, warning
+from pywikibot.tools import PY2
 
 OSWIN32 = (sys.platform == 'win32')
-PY2 = (sys.version_info[0] > 2)
 
 if OSWIN32:
 if not PY2:

-- 
To view, visit https://gerrit.wikimedia.org/r/239550
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id467fd3d0c176a70d9192639e694fd3b4fd2129a
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] editor: Close temporary file handle - change (pywikibot/core)

2015-09-19 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239561

Change subject: [FIX] editor: Close temporary file handle
..

[FIX] editor: Close temporary file handle

When using `mkstemp` the file handle must be closed before removing the file.
This was first fixed with 3a4d1844 but the `editor` module also uses `mkstemp`
and is not closing the handle.

Change-Id: I773aff3000fa26b04dcc9b5151fe11a726c0247c
---
M pywikibot/editor.py
1 file changed, 3 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/61/239561/1

diff --git a/pywikibot/editor.py b/pywikibot/editor.py
index 4ccf779..bef3119 100644
--- a/pywikibot/editor.py
+++ b/pywikibot/editor.py
@@ -95,7 +95,8 @@
 @rtype: unicode or None
 """
 if config.editor:
-tempFilename = '%s.%s' % (tempfile.mkstemp()[1],
+handle, tempFilename = tempfile.mkstemp()
+tempFilename = '%s.%s' % (tempFilename,
   config.editor_filename_extension)
 try:
 with codecs.open(tempFilename, 'w',
@@ -113,6 +114,7 @@
 newcontent = temp_file.read()
 return newcontent
 finally:
+os.close(handle)
 os.unlink(tempFilename)
 
 if isinstance(gui, ImportError):

-- 
To view, visit https://gerrit.wikimedia.org/r/239561
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I773aff3000fa26b04dcc9b5151fe11a726c0247c
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] Remove deprecation of returndict - change (pywikibot/core)

2015-09-19 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239558

Change subject: [FIX] Remove deprecation of returndict
..

[FIX] Remove deprecation of returndict

In bb29dafe the parameter `returndict` of `APISite.newpages` has been
deprecated and is replaced by nothing. But the function still supports that
parameter.

Change-Id: Iaadce9c3f8eca22c4fb9d7277b83b43eea66e655
---
M pywikibot/site.py
1 file changed, 0 insertions(+), 1 deletion(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/58/239558/1

diff --git a/pywikibot/site.py b/pywikibot/site.py
index 9aaceed..8aa221a 100644
--- a/pywikibot/site.py
+++ b/pywikibot/site.py
@@ -5767,7 +5767,6 @@
 @deprecated_args(number='total',
  repeat=None,
  namespace="namespaces",
- returndict=None,
  rcshow=None,
  rc_show=None,
  get_redirect=None)

-- 
To view, visit https://gerrit.wikimedia.org/r/239558
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Iaadce9c3f8eca22c4fb9d7277b83b43eea66e655
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] site_tests: Expect long if necessary - change (pywikibot/core)

2015-09-19 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/239566

Change subject: [FIX] site_tests: Expect long if necessary
..

[FIX] site_tests: Expect long if necessary

On the appveyor Windows builds, the maximum int number is to low to represent
the timestamp in an int and it uses a long. But the test explicitly requires an
int which is technically not possible.

Bug: T113146
Change-Id: I7c054c55cbb3334eed14faabdb09a69cbb18b48c
---
M tests/site_tests.py
1 file changed, 6 insertions(+), 2 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/66/239566/1

diff --git a/tests/site_tests.py b/tests/site_tests.py
index 2b41257..26227d0 100644
--- a/tests/site_tests.py
+++ b/tests/site_tests.py
@@ -22,8 +22,11 @@
 from pywikibot import config
 from pywikibot import async_request, page_put_queue
 from pywikibot.comms import http
-from pywikibot.tools import MediaWikiVersion
 from pywikibot.data import api
+from pywikibot.tools import (
+MediaWikiVersion,
+PY2,
+)
 
 from tests.aspects import (
 unittest, TestCase, DeprecationTestCase,
@@ -1062,7 +1065,8 @@
 self.assertIsInstance(entry, tuple)
 self.assertIsInstance(entry[0], pywikibot.Page)
 self.assertIsInstance(entry[1], basestring)
-self.assertIsInstance(entry[2], int)
+self.assertIsInstance(
+entry[2], long if PY2 and entry[2] > sys.maxint else int)  # 
noqa
 self.assertIsInstance(entry[3], basestring)
 
 def test_logpages_dump(self):

-- 
To view, visit https://gerrit.wikimedia.org/r/239566
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I7c054c55cbb3334eed14faabdb09a69cbb18b48c
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FEAT] site_detect_tests: Inform about duplicates - change (pywikibot/core)

2015-09-16 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/238701

Change subject: [FEAT] site_detect_tests: Inform about duplicates
..

[FEAT] site_detect_tests: Inform about duplicates

Several URLs have been added to the tests multiple times. And assuming each
test yields the same result running it multiple times won't help. This is
purposefully not cleared after each tests as the same URL may also be in
different tests.

Change-Id: Ifedd148a5662b416fe72db445f719860bd6a95c0
---
M tests/site_detect_tests.py
1 file changed, 18 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/01/238701/1

diff --git a/tests/site_detect_tests.py b/tests/site_detect_tests.py
index bcad8cb..4580f41 100644
--- a/tests/site_detect_tests.py
+++ b/tests/site_detect_tests.py
@@ -25,6 +25,11 @@
 
 """Test Case for MediaWiki detection and site object creation."""
 
+# This list is intentionally shared between all classes
+_urls_tested = set()
+# Whether it allows multiple tests of the same URL
+allow_multiple = False
+
 def setUp(self):
 """Set up test."""
 self.skips = {}
@@ -32,6 +37,8 @@
 self.errors = {}
 self.passes = {}
 self.all = []
+# reset after end of test
+self._previous_multiple = self.allow_multiple
 super(TestWikiSiteDetection, self).setUp()
 
 def tearDown(self):
@@ -51,6 +58,7 @@
 assert 0 <= pos < len(PREFIXES)
 return typ, url, res
 
+self.allow_multiple = self._previous_multiple
 super(TestWikiSiteDetection, self).tearDown()
 print('Out of %d sites, %d tests passed, %d tests failed, '
   '%d tests skiped and %d tests raised an error'
@@ -70,6 +78,14 @@
 def _wiki_detection(self, url, result):
 """Perform one load test."""
 self.all += [url]
+if url in self._urls_tested:
+msg = 'Testing URL "{0}" multiple times!'.format(url)
+if self.allow_multiple:
+print(msg)
+else:
+self.errors[url] = msg
+return
+self._urls_tested.add(url)
 try:
 site = MWSite(url)
 except (ServerError, Timeout) as e:
@@ -118,6 +134,8 @@
 code = 'meta'
 net = True
 
+allow_multiple = True
+
 def test_IWM(self):
 """Test the load_site method for MW sites on the IWM list."""
 data = self.get_site().siteinfo['interwikimap']

-- 
To view, visit https://gerrit.wikimedia.org/r/238701
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ifedd148a5662b416fe72db445f719860bd6a95c0
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

[MediaWiki-commits] [Gerrit] [FIX] doc: Fix syntax issues - change (pywikibot/core)

2015-09-16 Thread XZise (Code Review)

XZise has uploaded a new change for review.

  https://gerrit.wikimedia.org/r/238708

Change subject: [FIX] doc: Fix syntax issues
..

[FIX] doc: Fix syntax issues

Instead of using two single quotes like in wikitext it's using the reST's
syntax to write bold text using asterisks. Also separate lists by one newline
from text.

Change-Id: If2c86ba1ecce69b795646eb56d7104998f0dfeaa
---
M pywikibot/bot.py
M pywikibot/family.py
M pywikibot/page.py
3 files changed, 7 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core 
refs/changes/08/238708/1

diff --git a/pywikibot/bot.py b/pywikibot/bot.py
index 99a0d0f..104a45a 100644
--- a/pywikibot/bot.py
+++ b/pywikibot/bot.py
@@ -280,9 +280,10 @@
 user interfaces (GUIs) without modifying the core bot code.
 
 The following output levels are defined:
+
  - DEBUG: only for file logging; debugging messages.
  - STDOUT: output that must be sent to sys.stdout (for bots that may
- have their output redirected to a file or other destination).
+   have their output redirected to a file or other destination).
  - VERBOSE: optional progress information for display to user.
  - INFO: normal (non-optional) progress information for display to user.
  - INPUT: prompts requiring user response.
@@ -290,7 +291,7 @@
  - ERROR: user error messages.
  - CRITICAL: fatal error messages.
 
-Accordingly, do ''not'' use print statements in bot code; instead,
+Accordingly, do **not** use print statements in bot code; instead,
 use pywikibot.output function.
 
 @param strm: Output stream. If None, re-uses the last stream if one
@@ -1498,9 +1499,11 @@
 and puts the page if needed.
 
 Option used:
+
 * 'always'
 
 Keyword args used:
+
 * 'async' - passed to page.save
 * 'summary' - passed to page.save
 * 'show_diff' - show changes between oldtext and newtext (enabled)
diff --git a/pywikibot/family.py b/pywikibot/family.py
index 642698b..148a539 100644
--- a/pywikibot/family.py
+++ b/pywikibot/family.py
@@ -952,7 +952,7 @@
 
 Returns a string, not a compiled regular expression object.
 
-This reads from the family file, and ''not'' from
+This reads from the family file, and **not** from
 [[MediaWiki:Linktrail]], because the MW software currently uses a
 built-in linktrail from its message files and ignores the wiki
 value.
diff --git a/pywikibot/page.py b/pywikibot/page.py
index 180ad39..081c252 100644
--- a/pywikibot/page.py
+++ b/pywikibot/page.py
@@ -4386,6 +4386,7 @@
 Check whether the Claim's target is equal to specified value.
 
 The function checks for:
+
 - ItemPage ID equality
 - WbTime year equality
 - Coordinate equality, regarding precision

-- 
To view, visit https://gerrit.wikimedia.org/r/238708
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: If2c86ba1ecce69b795646eb56d7104998f0dfeaa
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: XZise 

___
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits

1 2 3 4 5 6 7 8 >

1 - 100 of 716 matches

Mail list logo