Re: [gentoo-portage-dev] [PATCH] runtests: rewrite in python
On Mon, 01 Jun 2015 23:16:47 +0200 Alexander Berntsen berna...@gentoo.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 30/05/15 20:27, Mike Frysinger wrote: to be clear: this is not new code. this is (more or less) a straight port from bash to python. your feedback here applies to the bash version as well. i'd like to minimize the changes when converting languages and then address feedback like this on top of that. I agree with Mike's approach here. - -- Alexander berna...@gentoo.org https://secure.plaimi.net/~alexander That's fine, commit the simple port, make other changes in later commits. -- Brian Dolbec dolsen
Re: [gentoo-portage-dev] [PATCH] runtests: rewrite in python
On 30 May 2015 12:30, Brian Dolbec wrote:
On Sat, 30 May 2015 14:27:25 -0400 Mike Frysinger wrote:
+def get_python_executable(ver):
+ Find the right python executable for |ver|
+ if ver == 'pypy':
+ prog = 'pypy'
+ else:
+ prog = 'python' + ver
+ return os.path.join(EPREFIX, 'usr', 'bin', prog)
The only thing I don't like about this is it could mean more
maintenance changes in the future if others come along.
to be clear: this is not new code. this is (more or less) a straight
port from bash to python. your feedback here applies to the bash
version as well. i'd like to minimize the changes when converting
languages and then address feedback like this on top of that.
I think making the lists have more complete naming would be better
PYTHON_SUPPORTED_VERSIONS = [
'python2.7',
'python3.3',
'python3.4',
]
# The rest are just nice to have.
PYTHON_NICE_VERSIONS = [
'pypy',
'python3.5',
'foo-bar-7.6',
]
Then all that is needed in get_python_executable() is the final
path. No other future code changes are likely to be needed.
Also easier to override from the environment without editing code.
this makes the command line interface a bit more annoying. today you
can do: $ runtests --python-version '2.7 3.3'
but with this change, it'd be:
$ runtests --python-version 'python2.7 python3.3'
we could add some logic so that if the arg is composed of dots
digits, we'd blindly prefix it with python.
Well, that get's back to almost the same kind of
get_python_executable(). But I think it would be a little better than
the existing.
We could instead do a string search and include any
member with that substring. That would include python3.3 and
foo-bar-3.3 for a 3.3 cli entry. It could make for a more flexible cli
versions = []
for y in args.python_versions:
versions.extend([x for x in all_pythons if y in x])
that would perform badly if you used 2 or 3 (as you'd match minors and
such). what about this patch ?
-mike
--- a/runtests
+++ b/runtests
@@ -15,20 +15,21 @@ from __future__ import print_function
import argparse
import os
+import re
import sys
import subprocess
# These are the versions we fully support and require to pass tests.
PYTHON_SUPPORTED_VERSIONS = [
- '2.7',
- '3.3',
- '3.4',
+ 'python2.7',
+ 'python3.3',
+ 'python3.4',
]
# The rest are just nice to have.
PYTHON_NICE_VERSIONS = [
'pypy',
- '3.5',
+ 'python3.5',
]
EPREFIX = os.environ.get('PORTAGE_OVERRIDE_EPREFIX', '/')
@@ -68,10 +69,10 @@ class Colors(object):
def get_python_executable(ver):
Find the right python executable for |ver|
- if ver == 'pypy':
- prog = 'pypy'
- else:
+ if re.match(r'[0-9.]+$', ver):
prog = 'python' + ver
+ else:
+ prog = ver
return os.path.join(EPREFIX, 'usr', 'bin', prog)
signature.asc
Description: Digital signature
Re: [gentoo-portage-dev] [PATCH] runtests: rewrite in python
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 30/05/15 20:27, Mike Frysinger wrote: to be clear: this is not new code. this is (more or less) a straight port from bash to python. your feedback here applies to the bash version as well. i'd like to minimize the changes when converting languages and then address feedback like this on top of that. I agree with Mike's approach here. - -- Alexander berna...@gentoo.org https://secure.plaimi.net/~alexander -BEGIN PGP SIGNATURE- Version: GnuPG v2 iQIcBAEBCgAGBQJVbMu+AAoJENQqWdRUGk8BAnAQAKiV2VOHq5FwoT2z32etGDnu lm0wv+4HbTMnLyy0PxIQBsyfM3X0Ee2obo2z5nNbmkj4MwvdzD0CJXvGJgdbo5Fq SbDp/NBZdMO4mjfuo1e3Bt3eTnRwAR+EG3BqpWanNmxGOQComTBKNI/lNOW/Vkaa zMWJ1ycK2svmDDKcx8NG1kQOJEo3be4k4cOHwV5zwqSwYYXEJFkApbdXYw861W29 6dAtRAWhn7MR41YqVAHoAjcnTcDb+hAr+DA8rzBk7/owX0J9TLEDTfIXm1klwcQ7 b3GGEto+EUXSBf2J5Kd7h4nWRIEIEfZpcXAsSbPGU66SORHurK0Ajeo17yX+m9Bi GSxi8dWtwQdjz4noQOtwXDrsTw6BCNpYuuiQYG+MshjKSgGFIRTcxH9FP5iNpj3G toqUFBr+RoBUmCTEnhJO7Dogt+xwHLDASJRksCBlDsG5OehmW7gE7pBctza+sDyy 14L/7XZB0mx8PGfVHIAcHXdjQ8Eu0yEn1BA/NfL5ZGuPtqRpqi7MrG5M3BrkHg8L 5I0d1SGohq28O5T8FSGy7CldvvZJliIhoxsRm+vY+lBKMcNi8fVvnmc1U42wggv4 ioxPb1+Xs2kJbu1QkdTjWMrivvFStSGVIsVoW2b1CejVP9lTGjfv0gY8FG+MHzod CuK6yOhzx3f1YI/j9Ehs =IGCH -END PGP SIGNATURE-
Re: [gentoo-portage-dev] [PATCH] runtests: rewrite in python
On Sat, May 30, 2015 at 1:18 PM, Brian Dolbec dol...@gentoo.org wrote: On Sat, 30 May 2015 12:29:14 -0400 Mike Frysinger vap...@gentoo.org wrote: The bash was getting ugly, and this allows us to add more smarts sanely to the main script. --- DEVELOPING | 2 +- runtests| 156 runtests.sh | 109 -- 3 files changed, 157 insertions(+), 110 deletions(-) create mode 100755 runtests delete mode 100755 runtests.sh diff --git a/DEVELOPING b/DEVELOPING index 0b0bb60..31b5594 100644 --- a/DEVELOPING +++ b/DEVELOPING @@ -225,7 +225,7 @@ Second create a git tag for this release: Then create the tarball and run the tests: ./mkrelease.sh --changelog-rev v2.2.7 --tag --runtests 2.2.8 Make sure you have all supported python versions installed first -(see PYTHON_SUPPORTED_VERSIONS in runtests.sh). +(see PYTHON_SUPPORTED_VERSIONS in runtests). Version bump the ebuild and verify it can re-install itself: emerge portage diff --git a/runtests b/runtests new file mode 100755 index 000..d1f7b6f --- /dev/null +++ b/runtests + +from __future__ import print_function If I'm not mistaken, this is not needed for 2.7+ anymore. It is still for 2.6 though. You are mistaken. Without this, print is still treated as a statement and anything in parens is treated as a tuple. % python2.7 --version Python 2.7.10 % python2.7 -c print(1, 2, 3) (1, 2, 3) % python2.7 -c from __future__ import print_function; print(1, 2, 3) 1 2 3
Re: [gentoo-portage-dev] [PATCH] runtests: rewrite in python
On 30 May 2015 14:42, Mike Gilbert wrote: On Sat, May 30, 2015 at 1:18 PM, Brian Dolbec wrote: On Sat, 30 May 2015 12:29:14 -0400 Mike Frysinger wrote: +from __future__ import print_function If I'm not mistaken, this is not needed for 2.7+ anymore. It is still for 2.6 though. You are mistaken. Without this, print is still treated as a statement and anything in parens is treated as a tuple. not exactly true -- (1) and (foo) are not tuples. need a comma to turn a paren into a tuple. e.g. (1,) and (foo,). -mike signature.asc Description: Digital signature
[gentoo-portage-dev] [PATCH] runtests: rewrite in python
The bash was getting ugly, and this allows us to add more smarts sanely
to the main script.
---
DEVELOPING | 2 +-
runtests| 156
runtests.sh | 109 --
3 files changed, 157 insertions(+), 110 deletions(-)
create mode 100755 runtests
delete mode 100755 runtests.sh
diff --git a/DEVELOPING b/DEVELOPING
index 0b0bb60..31b5594 100644
--- a/DEVELOPING
+++ b/DEVELOPING
@@ -225,7 +225,7 @@ Second create a git tag for this release:
Then create the tarball and run the tests:
./mkrelease.sh --changelog-rev v2.2.7 --tag --runtests 2.2.8
Make sure you have all supported python versions installed first
-(see PYTHON_SUPPORTED_VERSIONS in runtests.sh).
+(see PYTHON_SUPPORTED_VERSIONS in runtests).
Version bump the ebuild and verify it can re-install itself:
emerge portage
diff --git a/runtests b/runtests
new file mode 100755
index 000..d1f7b6f
--- /dev/null
+++ b/runtests
@@ -0,0 +1,156 @@
+#!/usr/bin/python
+# Copyright 2010-2015 Gentoo Foundation
+# Distributed under the terms of the GNU General Public License v2
+#
+# Note: We don't want to import portage modules directly because we do things
+# like run the testsuite through multiple versions of python.
+
+Helper script to run portage unittests against different python versions.
+
+Note: Any additional arguments will be passed down directly to the underlying
+unittest runner. This lets you select specific tests to execute.
+
+
+from __future__ import print_function
+
+import argparse
+import os
+import sys
+import subprocess
+
+
+# These are the versions we fully support and require to pass tests.
+PYTHON_SUPPORTED_VERSIONS = [
+ '2.7',
+ '3.3',
+ '3.4',
+]
+# The rest are just nice to have.
+PYTHON_NICE_VERSIONS = [
+ 'pypy',
+ '3.5',
+]
+
+EPREFIX = os.environ.get('PORTAGE_OVERRIDE_EPREFIX', '/')
+
+
+class Colors(object):
+ Simple object holding color constants.
+
+ _COLORS_YES = ('y', 'yes', 'true')
+ _COLORS_NO = ('n', 'no', 'false')
+
+ GOOD = BAD = NORMAL = ''
+
+ def __init__(self, colorize=None):
+ if colorize is None:
+ nocolors = os.environ.get('NOCOLOR', 'false')
+ # Ugh, look away, for here we invert the world!
+ if nocolors in self._COLORS_YES:
+ colorize = False
+ elif nocolors in self._COLORS_NO:
+ colorize = True
+ else:
+ raise ValueError('$NOCOLORS is invalid: %s' %
nocolors)
+ else:
+ if colorize in self._COLORS_YES:
+ colorize = True
+ elif colorize in self._COLORS_NO:
+ colorize = False
+ else:
+ raise ValueError('--colors is invalid: %s' %
colorize)
+
+ if colorize:
+ self.GOOD = '\033[1;32m'
+ self.BAD = '\033[1;31m'
+ self.NORMAL = '\033[0m'
+
+
+def get_python_executable(ver):
+ Find the right python executable for |ver|
+ if ver == 'pypy':
+ prog = 'pypy'
+ else:
+ prog = 'python' + ver
+ return os.path.join(EPREFIX, 'usr', 'bin', prog)
+
+
+def get_parser():
+ Return a argument parser for this module
+ epilog = Examples:
+List all the available unittests.
+$ %(prog)s --list
+
+Run against specific versions of python.
+$ %(prog)s --python-version '2.7 3.3'
+
+Run just one unittest.
+$ %(prog)s pym/portage/tests/xpak/test_decodeint.py
+
+ parser = argparse.ArgumentParser(
+ description=__doc__,
+ formatter_class=argparse.RawDescriptionHelpFormatter,
+ epilog=epilog)
+ parser.add_argument('--color', type=str, default=None,
+ help='Whether to use colorized output (default is auto)')
+ parser.add_argument('--python-versions', action='append',
+ help='Versions of python to test (default is test available)')
+ return parser
+
+
+def main(argv):
+ parser = get_parser()
+ opts, args = parser.parse_known_args(argv)
+ colors = Colors(colorize=opts.color)
+
+ # Figure out all the versions we want to test.
+ if opts.python_versions is None:
+ ignore_missing = True
+ pyversions = PYTHON_SUPPORTED_VERSIONS + PYTHON_NICE_VERSIONS
+ else:
+ ignore_missing = False
+ pyversions = []
+ for ver in opts.python_versions:
+ if ver == 'supported':
+ pyversions.extend(PYTHON_SUPPORTED_VERSIONS)
+ else:
+ pyversions.extend(ver.split())
+
+ #
Re: [gentoo-portage-dev] [PATCH] runtests: rewrite in python
On Sat, 30 May 2015 12:29:14 -0400 Mike Frysinger vap...@gentoo.org wrote: The bash was getting ugly, and this allows us to add more smarts sanely to the main script. --- DEVELOPING | 2 +- runtests| 156 runtests.sh | 109 -- 3 files changed, 157 insertions(+), 110 deletions(-) create mode 100755 runtests delete mode 100755 runtests.sh diff --git a/DEVELOPING b/DEVELOPING index 0b0bb60..31b5594 100644 --- a/DEVELOPING +++ b/DEVELOPING @@ -225,7 +225,7 @@ Second create a git tag for this release: Then create the tarball and run the tests: ./mkrelease.sh --changelog-rev v2.2.7 --tag --runtests 2.2.8 Make sure you have all supported python versions installed first -(see PYTHON_SUPPORTED_VERSIONS in runtests.sh). +(see PYTHON_SUPPORTED_VERSIONS in runtests). Version bump the ebuild and verify it can re-install itself: emerge portage diff --git a/runtests b/runtests new file mode 100755 index 000..d1f7b6f --- /dev/null +++ b/runtests + +from __future__ import print_function If I'm not mistaken, this is not needed for 2.7+ anymore. It is still for 2.6 though. +import argparse +import os +import sys +import subprocess + + +# These are the versions we fully support and require to pass tests. +PYTHON_SUPPORTED_VERSIONS = [ + '2.7', + '3.3', + '3.4', +] +# The rest are just nice to have. +PYTHON_NICE_VERSIONS = [ + 'pypy', + '3.5', +] + ... + + +def get_python_executable(ver): + Find the right python executable for |ver| + if ver == 'pypy': + prog = 'pypy' + else: + prog = 'python' + ver + return os.path.join(EPREFIX, 'usr', 'bin', prog) + + The only thing I don't like about this is it could mean more maintenance changes in the future if others come along. I think making the lists have more complete naming would be better PYTHON_SUPPORTED_VERSIONS = [ 'python2.7', 'python3.3', 'python3.4', ] # The rest are just nice to have. PYTHON_NICE_VERSIONS = [ 'pypy', 'python3.5', 'foo-bar-7.6', ] Then all that is needed in get_python_executable() is the final path. No other future code changes are likely to be needed. Also easier to override from the environment without editing code. PYTHON_NICE_VERSIONS=exp-py runtests... otherwise looks good. -- Brian Dolbec dolsen
Re: [gentoo-portage-dev] [PATCH] runtests: rewrite in python
On Sat, 30 May 2015 14:27:25 -0400 Mike Frysinger vap...@gentoo.org wrote: +def get_python_executable(ver): + Find the right python executable for |ver| + if ver == 'pypy': + prog = 'pypy' + else: + prog = 'python' + ver + return os.path.join(EPREFIX, 'usr', 'bin', prog) The only thing I don't like about this is it could mean more maintenance changes in the future if others come along. to be clear: this is not new code. this is (more or less) a straight port from bash to python. your feedback here applies to the bash version as well. i'd like to minimize the changes when converting languages and then address feedback like this on top of that. I think making the lists have more complete naming would be better PYTHON_SUPPORTED_VERSIONS = [ 'python2.7', 'python3.3', 'python3.4', ] # The rest are just nice to have. PYTHON_NICE_VERSIONS = [ 'pypy', 'python3.5', 'foo-bar-7.6', ] Then all that is needed in get_python_executable() is the final path. No other future code changes are likely to be needed. Also easier to override from the environment without editing code. this makes the command line interface a bit more annoying. today you can do: $ runtests --python-version '2.7 3.3' but with this change, it'd be: $ runtests --python-version 'python2.7 python3.3' we could add some logic so that if the arg is composed of dots digits, we'd blindly prefix it with python. -mike Well, that get's back to almost the same kind of get_python_executable(). But I think it would be a little better than the existing. We could instead do a string search and include any member with that substring. That would include python3.3 and foo-bar-3.3 for a 3.3 cli entry. It could make for a more flexible cli versions = [] for y in args.python_versions: versions.extend([x for x in all_pythons if y in x]) -- Brian Dolbec dolsen
