Hello community, here is the log from the commit of package python-ptr for openSUSE:Leap:15.2 checked in at 2020-03-20 05:15:14 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Leap:15.2/python-ptr (Old) and /work/SRC/openSUSE:Leap:15.2/.python-ptr.new.3160 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-ptr" Fri Mar 20 05:15:14 2020 rev:2 rq:786445 version:20.2.26 Changes: -------- --- /work/SRC/openSUSE:Leap:15.2/python-ptr/python-ptr.changes 2020-02-11 23:26:15.813296292 +0100 +++ /work/SRC/openSUSE:Leap:15.2/.python-ptr.new.3160/python-ptr.changes 2020-03-20 05:15:18.486564232 +0100 @@ -1,0 +2,13 @@ +Wed Mar 18 09:25:42 UTC 2020 - TomΓ‘Ε‘ ChvΓ‘tal <[email protected]> + +- Update to 20.2.26: + - Add support for passing `--system-site-packages` during venv creation - PR: #80 + - Add ability to create partial `.ptrconfig` files - PR: #83 + - Use native recursive file search for black and flake8 - PR: #81 + - Fix documentation for some new arguments - Thanks @omikader + - Windows now defaults to updating pip + setuptools + - Rely on setting current working directory with subprocess and remove coverage file environment setting + - f-string everything via [flynt](https://pypi.org/project/flynt/) now we're >= 3.6 + - Enable pyre for Python 3.8 now it's fully supported + +------------------------------------------------------------------- Old: ---- ptr-19.11.21.tar.gz New: ---- ptr-20.2.26.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-ptr.spec ++++++ --- /var/tmp/diff_new_pack.PPKPL8/_old 2020-03-20 05:15:19.546564943 +0100 +++ /var/tmp/diff_new_pack.PPKPL8/_new 2020-03-20 05:15:19.574564962 +0100 @@ -19,7 +19,7 @@ %{?!python_module:%define python_module() python-%{**} python3-%{**}} %define skip_python2 1 Name: python-ptr -Version: 19.11.21 +Version: 20.2.26 Release: 0 Summary: Parallel asyncio Python setup(cfg|py) test runner License: MIT ++++++ ptr-19.11.21.tar.gz -> ptr-20.2.26.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ptr-19.11.21/CHANGES.md new/ptr-20.2.26/CHANGES.md --- old/ptr-19.11.21/CHANGES.md 2019-11-22 04:51:01.000000000 +0100 +++ new/ptr-20.2.26/CHANGES.md 2020-02-27 01:05:45.000000000 +0100 @@ -2,6 +2,42 @@ Each release to PyPI I'm going to give a codename as to where I am or was in the world π. +### 2020.2.26 + +Codename: **Month after Straya Day π¦πΊ**πΊ + +*1 month since @cooperlees was in AU for Australia day!* + +- Enable pyre for Python 3.8 now it's fully supported + +### 2019.12.25 + +Codename: **Vernon Hills, IL π ** + +*@cooperlees visiting girlfriend's family in IL* + +- Fix documentation for some new arguments - Thanks @omikader +- Windows now defaults to updating pip + setuptools +- Rely on setting current working directory with subprocess and remove coverage file environment setting +- f-string everything via [flynt](https://pypi.org/project/flynt/) now we're >= 3.6 + +### 2019.12.13 + +Codename: **College Park, MD π’** + +*@omikader studied at the University of Maryland, College Park* + +- Add ability to create partial `.ptrconfig` files - PR: #83 +- Use native recursive file search for black and flake8 - PR: #81 + +### 2019.12.12 + +Codename: **Sapporo, Japan π―π΅** + +*@omikader is going skiing there in February* + +- Add support for passing `--system-site-packages` during venv creation - PR: #80 + ### 2019.11.21 Codename: **Santa Clara, CA** @@ -67,7 +103,7 @@ Codename: **Ellicott City** *@omikader who reported and tested the fix for AST parsing is from there.* -*Ellicott City is also home to one of the oldest surving train stations in the US!* +*Ellicott City is also home to one of the oldest surviving train stations in the US!* - Fix AST parsing to ignore AST targets that do not have an `id` attribute - Issue: #54 - Renable `black` by default on 3.7 now that it runs on > 3.7.2 - Issue: #41 @@ -97,7 +133,7 @@ *πΊπΈ @cooperlees + @jreese are at PyCon US in Cleveland, OH π¦ * -- Print step name in failue output - Issue: #31 +- Print step name in failure output - Issue: #31 - Add in `pyre` Type Checking step support - Issue: #38 + #40 **Known Bug:** `black.exe` does not run in Windows 3.7 - **disabled** by *default* on Python 3.7 on Windows diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ptr-19.11.21/PKG-INFO new/ptr-20.2.26/PKG-INFO --- old/ptr-19.11.21/PKG-INFO 2019-11-22 04:51:12.000000000 +0100 +++ new/ptr-20.2.26/PKG-INFO 2020-02-27 01:05:58.000000000 +0100 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: ptr -Version: 19.11.21 +Version: 20.2.26 Summary: Parallel asyncio Python setup.(cfg|py) Test Runner Home-page: http://github.com/facebookincubator/ptr Author: Cooper Lees @@ -65,7 +65,7 @@ usage: ptr.py [-h] [-a ATONCE] [-b BASE_DIR] [-d] [-e] [-k] [-m MIRROR] [--print-cov] [--print-non-configured] [--progress-interval PROGRESS_INTERVAL] [--run-disabled] - [--stats-file STATS_FILE] [--venv VENV] + [--stats-file STATS_FILE] [--system-site-packages] [--venv VENV] [--venv-timeout VENV_TIMEOUT] optional arguments: @@ -77,7 +77,8 @@ /Users/cooper/repos/ptr] -d, --debug Verbose debug output -e, --error-on-warnings - Have Python warnings raise DeprecationWarning + Have Python warnings raise DeprecationWarning on tests + run -k, --keep-venv Do not remove created venv -m MIRROR, --mirror MIRROR URL for pip to use for Simple API [Default: @@ -92,6 +93,9 @@ --stats-file STATS_FILE JSON statistics file [Default: /var/folders/tc/hbwxh76 j1hn6gqjd2n2sjn4j9k1glp/T/ptr_stats_12510] + --system-site-packages + Give the virtual environment access to the system + site-packages dir --venv VENV Path to venv to reuse --venv-timeout VENV_TIMEOUT Timeout in seconds for venv creation + deps install @@ -291,8 +295,6 @@ ### pyre - - `pyre` is not currently supported on 3.8. libCST requires [3.8 Support](https://github.com/Instagram/LibCST/issues/122) - ``` cooper-mbp1:ptr cooper$ /tmp/tp/bin/ptr --venv /var/folders/tc/hbwxh76j1hn6gqjd2n2sjn4j9k1glp/T/ptr_venv_49117 [2019-05-03 14:51:43,623] INFO: Starting /tmp/tp/bin/ptr (ptr.py:1023) @@ -335,7 +337,7 @@ ### Q. How do I get specific version of black, coverage, mypy etc.? - - Just simply hard set the version in the .ptrconfig in your repo + - Just simply hard set the version in the .ptrconfig in your repo or use `requirements.txt` to pre-install before running `ptr` - All `pip` [PEP 440 version specifiers](https://www.python.org/dev/peps/pep-0440/) are supported ### Q. Why is the venv creation so slow? @@ -351,6 +353,12 @@ - `pyre` (pyre-check on PyPI) does not ship a Windows wheel with the ocaml pyre.bin + ### Q. Why do you depend on >= coverage 5.0.1 + + - `coverage` 5.0 introduced using sqlite and we don't want to have a mix of 4.x and 5.x for ptr + - < 5.0 could possibly still work as we now ensure to run each projects tests from setup_py.parent CWD with subprocess + + # Contact or join the ptr community π¬ To chat in real time, hit us up on IRC. Otherwise, GitHub issues are always welcome! @@ -366,6 +374,42 @@ Each release to PyPI I'm going to give a codename as to where I am or was in the world π. + ### 2020.2.26 + + Codename: **Month after Straya Day π¦πΊ**πΊ + + *1 month since @cooperlees was in AU for Australia day!* + + - Enable pyre for Python 3.8 now it's fully supported + + ### 2019.12.25 + + Codename: **Vernon Hills, IL π ** + + *@cooperlees visiting girlfriend's family in IL* + + - Fix documentation for some new arguments - Thanks @omikader + - Windows now defaults to updating pip + setuptools + - Rely on setting current working directory with subprocess and remove coverage file environment setting + - f-string everything via [flynt](https://pypi.org/project/flynt/) now we're >= 3.6 + + ### 2019.12.13 + + Codename: **College Park, MD π’** + + *@omikader studied at the University of Maryland, College Park* + + - Add ability to create partial `.ptrconfig` files - PR: #83 + - Use native recursive file search for black and flake8 - PR: #81 + + ### 2019.12.12 + + Codename: **Sapporo, Japan π―π΅** + + *@omikader is going skiing there in February* + + - Add support for passing `--system-site-packages` during venv creation - PR: #80 + ### 2019.11.21 Codename: **Santa Clara, CA** @@ -431,7 +475,7 @@ Codename: **Ellicott City** *@omikader who reported and tested the fix for AST parsing is from there.* - *Ellicott City is also home to one of the oldest surving train stations in the US!* + *Ellicott City is also home to one of the oldest surviving train stations in the US!* - Fix AST parsing to ignore AST targets that do not have an `id` attribute - Issue: #54 - Renable `black` by default on 3.7 now that it runs on > 3.7.2 - Issue: #41 @@ -461,7 +505,7 @@ *πΊπΈ @cooperlees + @jreese are at PyCon US in Cleveland, OH π¦ * - - Print step name in failue output - Issue: #31 + - Print step name in failure output - Issue: #31 - Add in `pyre` Type Checking step support - Issue: #38 + #40 **Known Bug:** `black.exe` does not run in Windows 3.7 - **disabled** by *default* on Python 3.7 on Windows diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ptr-19.11.21/README.md new/ptr-20.2.26/README.md --- old/ptr-19.11.21/README.md 2019-11-22 04:51:01.000000000 +0100 +++ new/ptr-20.2.26/README.md 2020-02-27 01:05:45.000000000 +0100 @@ -57,7 +57,7 @@ usage: ptr.py [-h] [-a ATONCE] [-b BASE_DIR] [-d] [-e] [-k] [-m MIRROR] [--print-cov] [--print-non-configured] [--progress-interval PROGRESS_INTERVAL] [--run-disabled] - [--stats-file STATS_FILE] [--venv VENV] + [--stats-file STATS_FILE] [--system-site-packages] [--venv VENV] [--venv-timeout VENV_TIMEOUT] optional arguments: @@ -69,7 +69,8 @@ /Users/cooper/repos/ptr] -d, --debug Verbose debug output -e, --error-on-warnings - Have Python warnings raise DeprecationWarning + Have Python warnings raise DeprecationWarning on tests + run -k, --keep-venv Do not remove created venv -m MIRROR, --mirror MIRROR URL for pip to use for Simple API [Default: @@ -84,6 +85,9 @@ --stats-file STATS_FILE JSON statistics file [Default: /var/folders/tc/hbwxh76 j1hn6gqjd2n2sjn4j9k1glp/T/ptr_stats_12510] + --system-site-packages + Give the virtual environment access to the system + site-packages dir --venv VENV Path to venv to reuse --venv-timeout VENV_TIMEOUT Timeout in seconds for venv creation + deps install @@ -283,8 +287,6 @@ ### pyre -- `pyre` is not currently supported on 3.8. libCST requires [3.8 Support](https://github.com/Instagram/LibCST/issues/122) - ``` cooper-mbp1:ptr cooper$ /tmp/tp/bin/ptr --venv /var/folders/tc/hbwxh76j1hn6gqjd2n2sjn4j9k1glp/T/ptr_venv_49117 [2019-05-03 14:51:43,623] INFO: Starting /tmp/tp/bin/ptr (ptr.py:1023) @@ -327,7 +329,7 @@ ### Q. How do I get specific version of black, coverage, mypy etc.? -- Just simply hard set the version in the .ptrconfig in your repo +- Just simply hard set the version in the .ptrconfig in your repo or use `requirements.txt` to pre-install before running `ptr` - All `pip` [PEP 440 version specifiers](https://www.python.org/dev/peps/pep-0440/) are supported ### Q. Why is the venv creation so slow? @@ -343,6 +345,12 @@ - `pyre` (pyre-check on PyPI) does not ship a Windows wheel with the ocaml pyre.bin +### Q. Why do you depend on >= coverage 5.0.1 + +- `coverage` 5.0 introduced using sqlite and we don't want to have a mix of 4.x and 5.x for ptr +- < 5.0 could possibly still work as we now ensure to run each projects tests from setup_py.parent CWD with subprocess + + # Contact or join the ptr community π¬ To chat in real time, hit us up on IRC. Otherwise, GitHub issues are always welcome! diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ptr-19.11.21/ptr.egg-info/PKG-INFO new/ptr-20.2.26/ptr.egg-info/PKG-INFO --- old/ptr-19.11.21/ptr.egg-info/PKG-INFO 2019-11-22 04:51:12.000000000 +0100 +++ new/ptr-20.2.26/ptr.egg-info/PKG-INFO 2020-02-27 01:05:58.000000000 +0100 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: ptr -Version: 19.11.21 +Version: 20.2.26 Summary: Parallel asyncio Python setup.(cfg|py) Test Runner Home-page: http://github.com/facebookincubator/ptr Author: Cooper Lees @@ -65,7 +65,7 @@ usage: ptr.py [-h] [-a ATONCE] [-b BASE_DIR] [-d] [-e] [-k] [-m MIRROR] [--print-cov] [--print-non-configured] [--progress-interval PROGRESS_INTERVAL] [--run-disabled] - [--stats-file STATS_FILE] [--venv VENV] + [--stats-file STATS_FILE] [--system-site-packages] [--venv VENV] [--venv-timeout VENV_TIMEOUT] optional arguments: @@ -77,7 +77,8 @@ /Users/cooper/repos/ptr] -d, --debug Verbose debug output -e, --error-on-warnings - Have Python warnings raise DeprecationWarning + Have Python warnings raise DeprecationWarning on tests + run -k, --keep-venv Do not remove created venv -m MIRROR, --mirror MIRROR URL for pip to use for Simple API [Default: @@ -92,6 +93,9 @@ --stats-file STATS_FILE JSON statistics file [Default: /var/folders/tc/hbwxh76 j1hn6gqjd2n2sjn4j9k1glp/T/ptr_stats_12510] + --system-site-packages + Give the virtual environment access to the system + site-packages dir --venv VENV Path to venv to reuse --venv-timeout VENV_TIMEOUT Timeout in seconds for venv creation + deps install @@ -291,8 +295,6 @@ ### pyre - - `pyre` is not currently supported on 3.8. libCST requires [3.8 Support](https://github.com/Instagram/LibCST/issues/122) - ``` cooper-mbp1:ptr cooper$ /tmp/tp/bin/ptr --venv /var/folders/tc/hbwxh76j1hn6gqjd2n2sjn4j9k1glp/T/ptr_venv_49117 [2019-05-03 14:51:43,623] INFO: Starting /tmp/tp/bin/ptr (ptr.py:1023) @@ -335,7 +337,7 @@ ### Q. How do I get specific version of black, coverage, mypy etc.? - - Just simply hard set the version in the .ptrconfig in your repo + - Just simply hard set the version in the .ptrconfig in your repo or use `requirements.txt` to pre-install before running `ptr` - All `pip` [PEP 440 version specifiers](https://www.python.org/dev/peps/pep-0440/) are supported ### Q. Why is the venv creation so slow? @@ -351,6 +353,12 @@ - `pyre` (pyre-check on PyPI) does not ship a Windows wheel with the ocaml pyre.bin + ### Q. Why do you depend on >= coverage 5.0.1 + + - `coverage` 5.0 introduced using sqlite and we don't want to have a mix of 4.x and 5.x for ptr + - < 5.0 could possibly still work as we now ensure to run each projects tests from setup_py.parent CWD with subprocess + + # Contact or join the ptr community π¬ To chat in real time, hit us up on IRC. Otherwise, GitHub issues are always welcome! @@ -366,6 +374,42 @@ Each release to PyPI I'm going to give a codename as to where I am or was in the world π. + ### 2020.2.26 + + Codename: **Month after Straya Day π¦πΊ**πΊ + + *1 month since @cooperlees was in AU for Australia day!* + + - Enable pyre for Python 3.8 now it's fully supported + + ### 2019.12.25 + + Codename: **Vernon Hills, IL π ** + + *@cooperlees visiting girlfriend's family in IL* + + - Fix documentation for some new arguments - Thanks @omikader + - Windows now defaults to updating pip + setuptools + - Rely on setting current working directory with subprocess and remove coverage file environment setting + - f-string everything via [flynt](https://pypi.org/project/flynt/) now we're >= 3.6 + + ### 2019.12.13 + + Codename: **College Park, MD π’** + + *@omikader studied at the University of Maryland, College Park* + + - Add ability to create partial `.ptrconfig` files - PR: #83 + - Use native recursive file search for black and flake8 - PR: #81 + + ### 2019.12.12 + + Codename: **Sapporo, Japan π―π΅** + + *@omikader is going skiing there in February* + + - Add support for passing `--system-site-packages` during venv creation - PR: #80 + ### 2019.11.21 Codename: **Santa Clara, CA** @@ -431,7 +475,7 @@ Codename: **Ellicott City** *@omikader who reported and tested the fix for AST parsing is from there.* - *Ellicott City is also home to one of the oldest surving train stations in the US!* + *Ellicott City is also home to one of the oldest surviving train stations in the US!* - Fix AST parsing to ignore AST targets that do not have an `id` attribute - Issue: #54 - Renable `black` by default on 3.7 now that it runs on > 3.7.2 - Issue: #41 @@ -461,7 +505,7 @@ *πΊπΈ @cooperlees + @jreese are at PyCon US in Cleveland, OH π¦ * - - Print step name in failue output - Issue: #31 + - Print step name in failure output - Issue: #31 - Add in `pyre` Type Checking step support - Issue: #38 + #40 **Known Bug:** `black.exe` does not run in Windows 3.7 - **disabled** by *default* on Python 3.7 on Windows diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ptr-19.11.21/ptr.py new/ptr-20.2.26/ptr.py --- old/ptr-19.11.21/ptr.py 2019-11-22 04:51:01.000000000 +0100 +++ new/ptr-20.2.26/ptr.py 2020-02-27 01:05:45.000000000 +0100 @@ -22,12 +22,11 @@ from subprocess import CalledProcessError from tempfile import gettempdir from time import time -from typing import Any, Dict, Iterable, List, Optional, Sequence, Set, Tuple, Union +from typing import Any, Dict, List, Optional, Sequence, Set, Tuple, Union LOG = logging.getLogger(__name__) MACOSX = system() == "Darwin" -GREATER_THAN_37 = sys.version_info >= (3, 8) WINDOWS = system() == "Windows" # Windows needs to use a ProactorEventLoop for subprocesses # Need to use sys.platform for mypy to understand @@ -52,19 +51,19 @@ return cp -def _config_read(cwd: str, conf_name: str = ".ptrconfig") -> Optional[ConfigParser]: +def _config_read( + cwd: str, conf_name: str = ".ptrconfig", cp: ConfigParser = _config_default() +) -> ConfigParser: """ Look from cwd to / for a "conf_name" file - If so read it in """ - cp = None cwd_path = Path(cwd) # type: Path - root_path = Path("{}\\".format(cwd_path.drive)) if WINDOWS else Path("/") + root_path = Path(f"{cwd_path.drive}\\") if WINDOWS else Path("/") while cwd_path: ptrconfig_path = cwd_path / conf_name if ptrconfig_path.exists(): - cp = ConfigParser() cp.read(str(ptrconfig_path)) - LOG.info("Loading found config @ {}".format(ptrconfig_path)) + LOG.info(f"Loading found config @ {ptrconfig_path}") break if cwd_path == root_path: @@ -76,7 +75,7 @@ CWD = getcwd() -CONFIG = _config_read(CWD) or _config_default() +CONFIG = _config_read(CWD) PIP_CONF_TEMPLATE = """\ [global] index-url = {} @@ -113,7 +112,7 @@ if apath.is_dir() and apath.name == "site-packages": return apath - LOG.error("Unable to find a python lib dir in {}".format(lib_path)) + LOG.error(f"Unable to find a python lib dir in {lib_path}") return None @@ -140,7 +139,7 @@ ) return None if not required_cov: - LOG.error("No required coverage to enforce for {}".format(setup_py_path)) + LOG.error(f"No required coverage to enforce for {setup_py_path}") return None coverage_lines = {} @@ -168,9 +167,7 @@ if not module_path_str: LOG.error( - "[{}] Unable to find path relative path for {}".format( - setup_py_path, sl[0] - ) + f"[{setup_py_path}] Unable to find path relative path for {sl[0]}" ) continue @@ -185,10 +182,10 @@ if sl[0] != "TOTAL": stats[ - "suite.{}_coverage.file.{}".format(module_path.name, module_path_str) + f"suite.{module_path.name}_coverage.file.{module_path_str}" ] = coverage_lines[module_path_str].cover else: - stats["suite.{}_coverage.total".format(module_path.name)] = coverage_lines[ + stats[f"suite.{module_path.name}_coverage.total"] = coverage_lines[ module_path_str ].cover @@ -263,16 +260,12 @@ dump(stats, sfp, indent=2, sort_keys=True) except OSError as ose: LOG.exception( - "Unable to write out JSON statistics file to {} ({})".format( - stats_file, ose - ) + f"Unable to write out JSON statistics file to {stats_file} ({ose})" ) def _generate_black_cmd(module_dir: Path, black_exe: Path) -> Tuple[str, ...]: - py_files = set() # type: Set[str] - find_py_files(py_files, module_dir) - return (str(black_exe), "--check", *sorted(py_files)) + return (str(black_exe), "--check", ".") def _generate_install_cmd( @@ -296,7 +289,7 @@ module_dir: Path, mypy_exe: Path, config: Dict ) -> Tuple[str, ...]: if config.get("run_mypy", False): - mypy_entry_point = module_dir / "{}.py".format(config["entry_point_module"]) + mypy_entry_point = module_dir / f"{config['entry_point_module']}.py" else: return () @@ -314,14 +307,11 @@ if not config.get("run_flake8", False): return () - py_files = set() # type: Set[str] - find_py_files(py_files, module_dir) - cmds = [str(flake8_exe)] flake8_config = module_dir / ".flake8" if flake8_config.exists(): cmds.extend(["--config", str(flake8_config)]) - return (*cmds, *sorted(py_files)) + return tuple(cmds) def _generate_pylint_cmd( @@ -343,7 +333,7 @@ def _generate_pyre_cmd( module_dir: Path, pyre_exe: Path, config: Dict ) -> Tuple[str, ...]: - if not config.get("run_pyre", False) or WINDOWS or GREATER_THAN_37: + if not config.get("run_pyre", False) or WINDOWS: return () return (str(pyre_exe), "--source-directory", str(module_dir), "check") @@ -353,7 +343,7 @@ with setup_py.open("r", encoding="utf8") as sp: setup_tree = ast.parse(sp.read()) - LOG.debug("AST visiting {}".format(setup_py)) + LOG.debug(f"AST visiting {setup_py}") for node in ast.walk(setup_tree): if isinstance(node, ast.Assign): for target in node.targets: @@ -362,7 +352,7 @@ continue if target_id == "ptr_params": - LOG.debug("Found ptr_params in {}".format(setup_py)) + LOG.debug(f"Found ptr_params in {setup_py}") return dict(ast.literal_eval(node.value)) return {} @@ -385,9 +375,7 @@ non_configured_modules = [] # type: List[Path] test_modules = {} # type: Dict[Path, Dict] for setup_py in all_setup_pys: - disabled_err_msg = "Not running {} as ptr is disabled via config".format( - setup_py - ) + disabled_err_msg = f"Not running {setup_py} as ptr is disabled via config" # If a setup.cfg exists lets prefer it, if there is a [ptr] section ptr_params = parse_setup_cfg(setup_py) if not ptr_params: @@ -428,14 +416,14 @@ base_dir_path = Path(CWD) / base_dir_path if not base_dir_path.exists(): - LOG.error("{} does not exit. Not running tests".format(base_dir)) + LOG.error(f"{base_dir} does not exit. Not running tests") sys.exit(69) return base_dir_path async def _gen_check_output( - cmd: Iterable[str], + cmd: Sequence[str], timeout: Union[int, float] = 30, env: Optional[Dict[str, str]] = None, cwd: Optional[Path] = None, @@ -484,9 +472,7 @@ if not build_base_path or not build_base_path.exists(): if build_base_path: LOG.error( - "Configured local build env path {} does not exist".format( - build_base_path - ) + f"Configured local build env path {build_base_path} does not exist" ) return build_environ @@ -505,7 +491,7 @@ for var_name, value in build_env_vars: if var_name in build_environ: - build_environ[var_name] = "{}:{}".format(value, build_environ[var_name]) + build_environ[var_name] = f"{value}:{build_environ[var_name]}" else: build_environ[var_name] = str(value) else: @@ -538,13 +524,13 @@ ) -> Tuple[Optional[test_result], int]: bin_dir = "Scripts" if WINDOWS else "bin" exe = ".exe" if WINDOWS else "" - black_exe = venv_path / bin_dir / "black{}".format(exe) - coverage_exe = venv_path / bin_dir / "coverage{}".format(exe) - flake8_exe = venv_path / bin_dir / "flake8{}".format(exe) - mypy_exe = venv_path / bin_dir / "mypy{}".format(exe) - pip_exe = venv_path / bin_dir / "pip{}".format(exe) - pylint_exe = venv_path / bin_dir / "pylint{}".format(exe) - pyre_exe = venv_path / bin_dir / "pyre{}".format(exe) + black_exe = venv_path / bin_dir / f"black{exe}" + coverage_exe = venv_path / bin_dir / f"coverage{exe}" + flake8_exe = venv_path / bin_dir / f"flake8{exe}" + mypy_exe = venv_path / bin_dir / f"mypy{exe}" + pip_exe = venv_path / bin_dir / f"pip{exe}" + pylint_exe = venv_path / bin_dir / f"pylint{exe}" + pyre_exe = venv_path / bin_dir / f"pyre{exe}" config = tests_to_run[setup_py_path] steps = ( @@ -552,14 +538,14 @@ StepName.pip_install, True, _generate_install_cmd(str(pip_exe), str(setup_py_path.parent), config), - "Installing {} + deps".format(setup_py_path), + f"Installing {setup_py_path} + deps", config["test_suite_timeout"], ), step( StepName.tests_run, bool("test_suite" in config and config["test_suite"]), _generate_test_suite_cmd(coverage_exe, config), - "Running {} tests via coverage".format(config.get("test_suite", "")), + f"Running {config.get('test_suite', '')} tests via coverage", config["test_suite_timeout"], ), step( @@ -570,47 +556,42 @@ and len(config["required_coverage"]) > 0 ), (str(coverage_exe), "report", "-m"), - "Analyzing coverage report for {}".format(setup_py_path), + f"Analyzing coverage report for {setup_py_path}", config["test_suite_timeout"], ), step( StepName.mypy_run, bool("run_mypy" in config and config["run_mypy"]), _generate_mypy_cmd(setup_py_path.parent, mypy_exe, config), - "Running mypy for {}".format(setup_py_path), + f"Running mypy for {setup_py_path}", config["test_suite_timeout"], ), step( StepName.black_run, bool("run_black" in config and config["run_black"]), _generate_black_cmd(setup_py_path.parent, black_exe), - "Running black for {}".format(setup_py_path), + f"Running black for {setup_py_path}", config["test_suite_timeout"], ), step( StepName.flake8_run, bool("run_flake8" in config and config["run_flake8"]), _generate_flake8_cmd(setup_py_path.parent, flake8_exe, config), - "Running flake8 for {}".format(setup_py_path), + f"Running flake8 for {setup_py_path}", config["test_suite_timeout"], ), step( StepName.pylint_run, bool("run_pylint" in config and config["run_pylint"]), _generate_pylint_cmd(setup_py_path.parent, pylint_exe, config), - "Running pylint for {}".format(setup_py_path), + f"Running pylint for {setup_py_path}", config["test_suite_timeout"], ), step( StepName.pyre_run, - bool( - "run_pyre" in config - and config["run_pyre"] - and not WINDOWS - and not GREATER_THAN_37 - ), + bool("run_pyre" in config and config["run_pyre"] and not WINDOWS), _generate_pyre_cmd(setup_py_path.parent, pyre_exe, config), - "Running pyre for {}".format(setup_py_path), + f"Running pyre for {setup_py_path}", config["test_suite_timeout"], ), ) @@ -620,7 +601,7 @@ a_test_result = None # Skip test if disabled if not a_step.run_condition: - LOG.info("Not running {} step".format(a_step.log_message)) + LOG.info(f"Not running {a_step.log_message} step") continue LOG.info(a_step.log_message) @@ -628,24 +609,24 @@ steps_ran += 1 try: if a_step.cmds: - LOG.debug("CMD: {}".format(" ".join(a_step.cmds))) + LOG.debug(f"CMD: {' '.join(a_step.cmds)}") # If we're running tests and we want warnings to be errors step_env = env if a_step.step_name == StepName.tests_run and error_on_warnings: step_env = env.copy() step_env["PYTHONWARNINGS"] = "error" - LOG.debug("Setting PYTHONWARNINGS to error: {}".format(step_env)) + LOG.debug(f"Setting PYTHONWARNINGS to error: {step_env}") stdout, _stderr = await _gen_check_output( a_step.cmds, a_step.timeout, env=step_env, cwd=setup_py_path.parent ) else: - LOG.debug("Skipping running a cmd for {} step".format(a_step)) + LOG.debug(f"Skipping running a cmd for {a_step} step") except CalledProcessError as cpe: err_output = cpe.stdout.decode("utf8") - LOG.debug("{} FAILED for {}".format(a_step.log_message, setup_py_path)) + LOG.debug(f"{a_step.log_message} FAILED for {setup_py_path}") a_test_result = test_result( setup_py_path, a_step.step_name.value, @@ -654,15 +635,11 @@ False, ) except asyncio.TimeoutError as toe: - LOG.debug( - "{} timed out running {} ({})".format( - setup_py_path, a_step.log_message, toe - ) - ) + LOG.debug(f"{setup_py_path} timed out running {a_step.log_message} ({toe})") a_test_result = test_result( setup_py_path, a_step.step_name.value, - "Timeout during {}".format(a_step.log_message), + f"Timeout during {a_step.log_message}", a_step.timeout, True, ) @@ -670,7 +647,7 @@ if a_step.step_name is StepName.analyze_coverage: cov_report = stdout.decode("utf8") if stdout else "" if print_cov: - print("{}:\n{}".format(setup_py_path, cov_report)) + print(f"{setup_py_path}:\n{cov_report}") if a_step.run_condition: a_test_result = _analyze_coverage( @@ -698,24 +675,18 @@ error_on_warnings: bool, idx: int, ) -> None: - - # Set a unique location for coverage to write its data file per coroutine - cov_data_path = Path(gettempdir()) / "ptr.{}.{}.coverage".format(getpid(), idx) extra_build_env_path = ( Path(CONFIG["ptr"]["extra_build_env_prefix"]) if "extra_build_env_prefix" in CONFIG["ptr"] else None ) env = _set_build_env(extra_build_env_path) - env["COVERAGE_FILE"] = str(cov_data_path) while True: try: setup_py_path = queue.get_nowait() except asyncio.QueueEmpty: LOG.debug("test_runner {} exiting".format(idx)) - if cov_data_path.exists(): - cov_data_path.unlink() return test_run_start_time = int(time()) @@ -733,7 +704,7 @@ if test_fail_result: test_results.append(test_fail_result) else: - success_output = "{} has passed all configured tests".format(setup_py_path) + success_output = f"{setup_py_path} has passed all configured tests" LOG.info(success_output) test_results.append( test_result( @@ -742,8 +713,8 @@ ) stats_name = setup_py_path.parent.name - stats["suite.{}_runtime".format(stats_name)] = total_success_runtime - stats["suite.{}_completed_steps".format(stats_name)] = steps_ran + stats[f"suite.{stats_name}_runtime"] = total_success_runtime + stats[f"suite.{stats_name}_completed_steps"] = steps_ran queue.task_done() @@ -753,9 +724,10 @@ py_exe: str = sys.executable, install_pkgs: bool = True, timeout: float = VENV_TIMEOUT, + system_site_packages: bool = False, ) -> Optional[Path]: start_time = time() - venv_path = Path(gettempdir()) / "ptr_venv_{}".format(getpid()) + venv_path = Path(gettempdir()) / f"ptr_venv_{getpid()}" if WINDOWS: pip_exe = venv_path / "Scripts" / "pip.exe" else: @@ -763,30 +735,26 @@ install_cmd: List[str] = [] try: - await _gen_check_output((py_exe, "-m", "venv", str(venv_path)), timeout=timeout) + cmd = [py_exe, "-m", "venv", str(venv_path)] + if system_site_packages: + cmd.append("--system-site-packages") + + await _gen_check_output(cmd, timeout=timeout) _set_pip_mirror(venv_path, mirror) if install_pkgs: install_cmd = [str(pip_exe), "install"] - # TODO: Workout why Windows can't uninstall / upgrade pip on Azure - # https://github.com/facebookincubator/ptr/issues/33 - if not WINDOWS: - install_cmd.append("--upgrade") install_cmd.extend(CONFIG["ptr"]["venv_pkgs"].split()) await _gen_check_output(install_cmd, timeout=timeout) except CalledProcessError as cpe: - LOG.exception( - "Failed to setup venv @ {} - '{}'' ({})".format(venv_path, install_cmd, cpe) - ) + LOG.exception(f"Failed to setup venv @ {venv_path} - '{install_cmd}'' ({cpe})") if cpe.stderr: - LOG.debug("venv stderr:\n{}".format(cpe.stderr.decode("utf8"))) + LOG.debug(f"venv stderr:\n{cpe.stderr.decode('utf8')}") if cpe.output: - LOG.debug("venv stdout:\n{}".format(cpe.output.decode("utf8"))) + LOG.debug(f"venv stdout:\n{cpe.output.decode('utf8')}") return None runtime = int(time() - start_time) - LOG.info( - "Successfully created venv @ {} to run tests ({}s)".format(venv_path, runtime) - ) + LOG.info(f"Successfully created venv @ {venv_path} to run tests ({runtime}s)") return venv_path @@ -815,9 +783,7 @@ if directory.match(exclude_pattern): skip_dir = True LOG.debug( - "Skipping {} due to exclude pattern {}".format( - directory, exclude_pattern - ) + f"Skipping {directory} due to exclude pattern {exclude_pattern}" ) if not skip_dir: _recursive_find_files(files, directory, exclude_patterns, follow_symlinks) @@ -863,9 +829,9 @@ def print_non_configured_modules(modules: List[Path]) -> None: - print("== {} non ptr configured modules ==".format(len(modules))) + print(f"== {len(modules)} non ptr configured modules ==") for module in sorted(modules): - print(" - {}".format(str(module))) + print(f" - {str(module)}") def print_test_results( @@ -890,7 +856,7 @@ stats["total.passes"] += 1 total_time = -1 if "runtime.all_tests" not in stats else stats["runtime.all_tests"] - print("-- Summary (total time {}s):\n".format(total_time)) + print(f"-- Summary (total time {total_time}s):\n") # TODO: Hardcode some workaround to ensure Windows always prints UTF8 # https://github.com/facebookincubator/ptr/issues/34 print( @@ -932,12 +898,17 @@ stats_file: str, venv_timeout: float, error_on_warnings: bool, + system_site_packages: bool, ) -> int: tests_start_time = time() if not venv_path or not venv_path.exists(): venv_create_start_time = time() - venv_path = await create_venv(mirror=mirror, timeout=venv_timeout) + venv_path = await create_venv( + mirror=mirror, + timeout=venv_timeout, + system_site_packages=system_site_packages, + ) stats["venv_create_time"] = int(time() - venv_create_start_time) else: venv_keep = True @@ -969,9 +940,7 @@ for i in range(atonce) ] if progress_interval: - LOG.debug( - "Adding progress reporter to report every {}s".format(progress_interval) - ) + LOG.debug(f"Adding progress reporter to report every {progress_interval}s") consumers.append( _progress_reporter(progress_interval, queue, len(tests_to_run)) ) @@ -987,7 +956,7 @@ chdir(gettempdir()) rmtree(str(venv_path)) else: - LOG.info("Not removing venv @ {} due to CLI arguments".format(venv_path)) + LOG.info(f"Not removing venv @ {venv_path} due to CLI arguments") return stats["total.fails"] + stats["total.timeouts"] @@ -1005,6 +974,7 @@ stats_file: str, venv_timeout: float, error_on_warnings: bool, + system_site_packages: bool, ) -> int: stats = defaultdict(int) # type: Dict[str, int] tests_to_run = _get_test_modules( @@ -1012,9 +982,7 @@ ) if not tests_to_run: LOG.error( - "{} has no setup.py files with unit tests defined. Exiting".format( - str(base_path) - ) + f"{str(base_path)} has no setup.py files with unit tests defined. Exiting" ) return 1 @@ -1024,7 +992,7 @@ try: venv_path = Path(venv) # type: Optional[Path] if venv_path and not venv_path.exists(): - LOG.error("{} venv does not exist. Please correct!".format(venv_path)) + LOG.error(f"{venv_path} venv does not exist. Please correct!") return 2 except TypeError: venv_path = None @@ -1041,26 +1009,25 @@ stats_file, venv_timeout, error_on_warnings, + system_site_packages, ) def main() -> None: - default_stats_file = Path(gettempdir()) / "ptr_stats_{}".format(getpid()) + default_stats_file = Path(gettempdir()) / f"ptr_stats_{getpid()}" parser = argparse.ArgumentParser() parser.add_argument( "-a", "--atonce", default=int(CONFIG["ptr"]["atonce"]), type=int, - help="How many tests to run at once [Default: {}]".format( - int(CONFIG["ptr"]["atonce"]) - ), + help=f"How many tests to run at once [Default: {int(CONFIG['ptr']['atonce'])}]", ) parser.add_argument( "-b", "--base-dir", default=CWD, - help="Path to recursively look for setup.py files [Default: {}]".format(CWD), + help=f"Path to recursively look for setup.py files [Default: {CWD}]", ) parser.add_argument( "-d", "--debug", action="store_true", help="Verbose debug output" @@ -1104,7 +1071,12 @@ parser.add_argument( "--stats-file", default=str(default_stats_file), - help="JSON statistics file [Default: {}]".format(default_stats_file), + help=f"JSON statistics file [Default: {default_stats_file}]", + ) + parser.add_argument( + "--system-site-packages", + action="store_true", + help="Give the virtual environment access to the system site-packages dir", ) parser.add_argument("--venv", help="Path to venv to reuse") parser.add_argument( @@ -1118,7 +1090,7 @@ args = parser.parse_args() _handle_debug(args.debug) - LOG.info("Starting {}".format(sys.argv[0])) + LOG.info(f"Starting {sys.argv[0]}") loop = asyncio.get_event_loop() try: sys.exit( @@ -1136,6 +1108,7 @@ args.stats_file, args.venv_timeout, args.error_on_warnings, + args.system_site_packages, ) ) ) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ptr-19.11.21/ptr_tests.py new/ptr-20.2.26/ptr_tests.py --- old/ptr-19.11.21/ptr_tests.py 2019-11-22 04:51:01.000000000 +0100 +++ new/ptr-20.2.26/ptr_tests.py 2020-02-27 01:05:45.000000000 +0100 @@ -19,6 +19,7 @@ Dict, List, Optional, + Sequence, Tuple, ) from unittest.mock import Mock, patch @@ -37,6 +38,10 @@ return None +async def check_site_package_config(cmd: Sequence, *args: Any, **kwargs: Any) -> None: + assert "--system-site-packages" in cmd, f"--system-site-packages not found in {cmd}" + + def fake_get_event_loop(*args: Any, **kwargs: Any) -> ptr_tests_fixtures.FakeEventLoop: return ptr_tests_fixtures.FakeEventLoop() @@ -55,6 +60,10 @@ return 2580217 +def return_zero(*args: Any, **kwargs: Any) -> int: + return 0 + + def touch_files(*paths: Path) -> None: for path in paths: path.parent.mkdir(parents=True, exist_ok=True) @@ -141,7 +150,7 @@ def test_mac_osx_slash_private(self) -> None: macosx = ptr.MACOSX non_private_path_str = "/var/tmp" - private_path_str = "/private{}".format(non_private_path_str) + private_path_str = f"/private{non_private_path_str}" site_packages_path = Path("/var/tmp/venv/lib/site-packages/") try: ptr.MACOSX = False @@ -181,6 +190,7 @@ "stats", 30, True, + False, ] mock_gtm.return_value = False self.assertEqual( @@ -204,9 +214,7 @@ td = Path(__file__).parent sc = ptr._config_read(str(td), "ptrconfig.sample") - self.assertEqual( - sc["ptr"].get("pypi_url", ""), expected_pypi_url # pyre-ignore - ) + self.assertEqual(sc["ptr"].get("pypi_url", ""), expected_pypi_url) self.assertEqual(len(sc["ptr"].get("venv_pkgs", "").split()), 8) @patch("ptr._gen_check_output", async_none) @@ -218,6 +226,15 @@ ) ) + @patch("ptr._gen_check_output", check_site_package_config) + @patch("ptr._set_pip_mirror") + def test_create_venv_site_packages(self, mock_pip_mirror: Mock) -> None: + self.loop.run_until_complete( + ptr.create_venv( + "https://pip.com/";, install_pkgs=False, system_site_packages=True + ) + ) + def test_find_setup_py(self) -> None: base_path = Path(__file__).parent found_setup_py = ptr.find_setup_pys(base_path, set()).pop() @@ -241,14 +258,10 @@ black_exe = Path("/bin/black") with TemporaryDirectory() as td: module_dir = Path(td) - subdir = module_dir / "awlib" - py2 = subdir / "awesome2.py" - py1 = module_dir / "awesome.py" - touch_files(py1, py2) self.assertEqual( ptr._generate_black_cmd(module_dir, black_exe), - (str(black_exe), "--check", str(py1), str(py2)), + (str(black_exe), "--check", "."), ) def test_generate_install_cmd(self) -> None: @@ -291,17 +304,14 @@ flake8_exe = Path("/bin/flake8") with TemporaryDirectory() as td: module_dir = Path(td) - subdir = module_dir / "awlib" cf = module_dir / ".flake8" - py2 = subdir / "awesome2.py" - py1 = module_dir / "awesome.py" - touch_files(cf, py1, py2) + touch_files(cf) conf = {"run_flake8": True} self.assertEqual( ptr._generate_flake8_cmd(module_dir, flake8_exe, conf), - (str(flake8_exe), "--config", str(cf), str(py1), str(py2)), + (str(flake8_exe), "--config", str(cf)), ) def test_generate_pylint_command(self) -> None: @@ -328,7 +338,7 @@ conf = {"run_pyre": True} expected = (str(pyre_exe), "--source-directory", str(td_path), "check") - if ptr.WINDOWS or ptr.GREATER_THAN_37: + if ptr.WINDOWS: expected = () self.assertEqual(ptr._generate_pyre_cmd(td_path, pyre_exe, conf), expected) @@ -376,12 +386,10 @@ self.assertEqual(ptr._handle_debug(True), True) @patch("ptr.asyncio.get_event_loop", fake_get_event_loop) - @patch("ptr.async_main") + @patch("ptr.async_main", return_zero) @patch("ptr._validate_base_dir") @patch("ptr.argparse.ArgumentParser.parse_args") - def test_main( - self, mock_args: Mock, mock_validate: Mock, mock_async_main: Mock - ) -> None: + def test_main(self, mock_args: Mock, mock_validate: Mock) -> None: with self.assertRaises(SystemExit): ptr.main() @@ -478,8 +486,8 @@ fake_venv_path = td_path / "unittest_venv" fake_venv_lib_path = fake_venv_path / "lib" fake_venv_lib_path.mkdir(parents=True) - # Windows + Python 3.8 will not run pyre - no_pyre = ptr.WINDOWS or ptr.GREATER_THAN_37 + # Windows can not run pyre + no_pyre = ptr.WINDOWS tsr_params = [ 69, # test_start_time {fake_setup_py: {}}, # tests_to_run diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ptr-19.11.21/ptr_tests_fixtures.py new/ptr-20.2.26/ptr_tests_fixtures.py --- old/ptr-19.11.21/ptr_tests_fixtures.py 2019-11-22 04:51:01.000000000 +0100 +++ new/ptr-20.2.26/ptr_tests_fixtures.py 2020-02-27 01:05:45.000000000 +0100 @@ -42,7 +42,7 @@ returncode=3, output=( "The following files did not meet coverage requirements:\n" - + " unittest{}ptr.py: 69 < 99 - Missing: 70-72, 76-94, 98\n".format(sep) + + f" unittest{sep}ptr.py: 69 < 99 - Missing: 70-72, 76-94, 98\n" ), runtime=0, timeout=False, @@ -51,9 +51,7 @@ setup_py_path=Path("unittest/setup.py"), returncode=3, output=( - "The following files did not meet coverage requirements:\n tg{}tg.py: ".format( - sep - ) + f"The following files did not meet coverage requirements:\n tg{sep}tg.py: " + "22 < 99 - Missing: 39-59, 62-73, 121, 145-149, 153-225, 231-234, 238\n " + "TOTAL: 40 < 99 - Missing: \n" ), @@ -111,7 +109,7 @@ sep=sep ) -HARD_SET_VENV = Path("{}/ptr_venv_2580217".format(gettempdir())) +HARD_SET_VENV = Path(f"{gettempdir()}/ptr_venv_2580217") BASE_VENV_PATH = ( Path(environ["VIRTUAL_ENV"]) if "VIRTUAL_ENV" in environ else HARD_SET_VENV ) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/ptr-19.11.21/setup.py new/ptr-20.2.26/setup.py --- old/ptr-19.11.21/setup.py 2019-11-22 04:51:01.000000000 +0100 +++ new/ptr-20.2.26/setup.py 2020-02-27 01:05:45.000000000 +0100 @@ -47,7 +47,7 @@ setup( name=ptr_params["entry_point_module"], - version="19.11.21", + version="20.2.26", description="Parallel asyncio Python setup.(cfg|py) Test Runner", long_description=get_long_desc(), long_description_content_type="text/markdown",
